Experimental Physics and
Industrial Control System

Benjamin Franksen <[email protected]> · Thu, 27 Oct 2005 22:34:20 +0200

On Thursday 27 October 2005 19:03, Rees, NP (Nick) wrote:
> 2. My trivial example would probably not be like Kay's, I would put
> all the information in attributes, not as free-form data. 

Yes, definitely. When I first learned about XML and especially XSD I was 
fascinated by the idea to use it to replace the db/dbd file format. I 
never talked about it much (it would surely have been dismissed as 
"nice, but what does it buy us?", back then), anyway, I came up with 
something very similar to your example.

> I think this fits the V3 db definition more cleanly (i.e. the
> converter is probably a few lines of awk) and in fact I don't think
> this is really any less readable than the current format. 

Agreed. However, it /is/ more verbose which makes editing it with an 
editor more cumbersome. Not too much, I hope.

> However, 
> the bonus for me is that I can define extra tags that I know will be
> ignored by systems that don't understand them as long as they
> validate. My problem is really that the info item just is to simple
> to be of much use.

Yes, yes, yes! Unstructured strings are almost never enough to provide 
real extensibility. Witness the hardware link desaster in EPICS. Let us 
have extensible syntax and data types. XML/XSD gives us the framework.

> 3. As to Kay's "well, why not XML" point, I agree that this is often
> a reason, but at heart this is because we are now accepting that we
> don't have to write parsers anymore and/or use lex and yacc. Overall,
> I believe this is a good thing. 

Yes. Absolutely.

> In my experience this turns into a 
> bigger advantage when these files essentially become interface
> definitions between isolated components, and we have a schema which
> automatically validates the interface. This eliminates a lot of
> discussion late on in a large, distributed project. So, I don't
> expect users to come into my office to thank me for using XML. I
> don't expect my users to even know I'm using XML.

I am not so sure about that. There will always be a need to manually 
edit configuration files. The flexibility you get with a straight text 
editor is unsurpassed: I know of no specialized tool in the world with 
which you can do arbitrary pattern based search-and-replace not only on 
data fields but on structure elements, and still *see* what you are 
doing while you do it. And I very much remember what a disaster for 
usability the obscure central 'registry' was when MS introduced Win95, 
as opposed to the simple and easy-to-edit dot-ini files.

We should take care that manual editing with a simple text editor will 
not be prohibitively tedious.

(BTW, one of the features we would like for VDCT is to make bulk editing 
of properties easier. I want to select multiple records, bring up the 
property window which displays all fields common to the selected 
records (which can be of different types, i.e. intersection of 
properties; or optionally also the union); editing would change the 
value for all selected records, etc. This would greatly reduce, though 
probably not completely obviate, the need to use a text editor.)

> 4. Finally, I accept Pam's point that the latest version of most of
> this information should reside in a relational database. However,
> systems often require information to be stored in many forms - and a
> textual form is usually always one of them. This is the subject I am
> trying to address. Lets face it, a lot of the code we write is just
> converting information into different formats. If you compare text
> and relational databases it is more difficult to use source code
> control on relational databases, 

Yes. We have been fighting with this at BESSY-II ever since we started 
to use Oracle for storing device configuraton data. Proper versioning 
like in CVS is still an open problem for us. I can easily checkout an 
old version of our control system sources from CVS, but whether it 
still builds (not to mention works) depends a lot on what has changed 
in our RDB.

However:

> and you still need a way of encoding 
> data for transmission between applications - this is easier to debug
> if it is textual. 

Yes. And using XML /might/ have the advantage of easier integration with 
RDBs. That is, creating an XML image of some RDB view, editing it, and 
later putting the whole thing back into the RDB. This is important 
because nobody expects, for instance, VDCT to aquire an Oracle backend 
any time soon ;-)

The real challenge here is how to keep all the primary keys valid, i.e. 
restore into the RDB in such a way that the identity of previously 
existing elements (records, fields, ...) in the RDB remain the same 
(they might be referenced from somewhere else in the RDB). With an 
extensible file format we could add <key> tags that get ignored by most 
tools (but preserved by editing tools as far as possible).

> As a current example, the latest stuff Matej has 
> done on VDCT has involved cutting and pasting databases in text form
> via the clipboard. 

Cool!

> This will allow Don Dohan to copy an EPICS 
> database out of his relational database onto the system clipboard and
> then paste it into VDCT so he can see how the links work. If you
> follow this path a little way, we might be able to do the same for
> the wiring information in Don's database - if we agree on a common
> schema, VDCT could start plotting the 120 volt power hookups as well.
> I am just dreaming here, but in doing so I am trying to convey the
> reason. This is not just about the definition of IOC databases
> anymore, I am trying to invent a world where we have a fairly
> flexible (i.e. extensible) data interchange format of which the IOC
> database is only a part. I hope this also addresses Kay's concern
> that XML hasn't made his life easier. I really starts leveraging
> itself if you use it as an information interchange format that we can
> all agree on. Using just for configuration of a single application is
> not its main justification.

I completely agree with everything, here.

Let me finally add that I think making life easier for the developers is 
ultimately also better for the users: requests for changes (mostly: 
additional features) will be met with less reluctance (like "Oh, I 
could surely do this, but it's not *that* easy; I would need x 
days/weeks to make sure it works; do you *really* need that?").

Cheers,
Ben

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System