EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  <20102011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  <20102011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
<== Date ==> <== Thread ==>

Subject: Re: msi again
From: Benjamin Franksen <[email protected]>
To: [email protected]
Date: Tue, 24 Aug 2010 14:27:06 +0200
On Monday, August 23, 2010, you wrote:
> I'd like to agree on a
> formal grammar for the substitutions file.

Yay!

> Having looked through the Yacc grammar for dbLoadTemplate I should point
> out that it doesn't currently support empty patterns or substitutions. 
> With the addition of the global variables I think it makes sense to
> allow this: file xyz {}
> but I don't see why you'd need to be able to write either of these as
> well: file xyz {{}}
>   file xyz { pattern {} {} }

So, let us talk about semantics before we formalize the syntax. As I 
understand it,

  file <filename> { <list-of-substitution-lists> }

should mean:

  For each substitution-list in <list-of-substitution-lists> insert
  content of file named <filename> after performing all substitutions
  in scope.

The substitutions in scope inside such a file block are defined to be:

  All global substitutions preceding the file block (where later
  definitions for the same macro override earlier ones) and additionally
  all substitutions defined for the instance, again with later
  definitions shadowing earlier ones.

We have two kinds of lists here that may be empty. By natural generalization 
of the above definition

* an empty list-of-substitution-lists ("file xyz {}") means: do not insert
  anything.
* an empty list of substitutions means insert content of file with only
  the global substitutions in scope (if any), thus "file xyz { {} {} }"
  would insert the contents of file xyz twice.

Both are occasionally useful, especially when generating or processing 
substitution files. Note that while msi is primarily meant to expand 
database templates, it is by design *not* restricted to this task. There may 
be use cases where inserting the same text more than once make more sense 
than it does with record instances.

> Here's what I think the syntax should look like with "global" added,
> expressed in EBNF:
> 
> 
> substitution-file ::= ( global-definitions* template-substitutions+ )+

This looks unnecessarily restrictive to me. Why not simply

  substitution-file ::= ( global-definitions | template-substitutions )*

Even an empty substitution file is not completely unreasonable (no file 
sections means output nothing). Note that none of the "empty" cases I 
propose need special-case semantics, they are pure straight-forward 
generalizations.

> global-definitions ::= 'global' '{' variable-definitions '}'
> variable-definitions ::= variable-definition ( ',' variable-definition )*

The comma between definitions is not always required currently (in msi). I 
propose to make it strictly optional. This would make generating and writing 
substitution files a lot more fool-proof. Note that "," (comma) is not a 
value-char, so is not allowed in unquoted values. In fact, ',' does not 
begin or end any token (except ',' itself), so there is no ambiguity:

  x=1,y=2

gets parsed as

  x="1", y="2"

I also (see above) want empty definition-lists, so my (counter-)proposal 
here is:

  variable-definitions ::= ( variable-definition ','? )*

[In principle one could treat ',' (except inside quoted strings) everywhere 
as whitespace, but I'd rather limit it to delimiting list elements. It might 
make sense to allow it after file/global blocks and after substitution-
lists, just for consistency.]

> variable-definition ::= variable-name '=' value
> 
> template-substitutions ::= 'file' file-name '{' substitutions? '}'

I'd rather push down the empty case into the variable-substitutions part 
(because the latter have a natural empty case, whereas a pattern-definition 
must start with a pattern-names clause). Thus

  template-substitutions ::= 'file' file-name '{' substitutions '}'

> substitutions ::= variable-substitutions | pattern-substitutions
> variable-substitutions ::= '{' variable-definitions '}'

This should be

  variable-substitutions ::= ( '{' variable-definitions '}' )*

> pattern-substitutions ::= 'pattern' '{' pattern-names '}'
>   pattern-definitions 
> pattern-names ::= variable-name ( ',' variable-name )*

See my comments wrt variable-definitions, i.e. this should be

  pattern-names ::= ( variable-name ','? )*

> pattern-definitions ::= ( '{' pattern-values '}' )+

Again, I see no reason to exclude the empty case, so

  pattern-definitions ::= ( '{' pattern-values '}' )*

> pattern-values ::= value ( ',' value )*

See my comments wrt variable-definitions, i.e. this should be

  pattern-values ::= ( value ','? )*

> variable-name ::= variable-name-char+
> file-name ::= file-name-char+ | double-quoted-string |
>   single-quoted-string
> value ::= value-char+ | double-quoted-string |
>   single-quoted-string
> 
> double-quoted-string ::= '"' (double-quoted-char | escaped-char)* '"'
> single-quoted-string ::= "'" (single-quoted-char | escaped-char)* "'"
> double-quoted-char ::= [^"\]
> single-quoted-char ::= [^'\]
> escaped-char ::= '\' .
> 
> Currently dbLoadTemplate_lex.l defines the remaining terminals to be
>   variable-name-char ::= value-char
>   file-name-char ::= value-char
>   value-char ::= [a-zA-Z0-9_+:./\<>;[] | '-' | ']'
> 
> I would prefer something like these instead
>   variable-name-char ::= [a-zA-Z0-9_]
>   file-name-char ::= [a-zA-Z0-9_+:./\] | '-'
>   value-char ::= [a-zA-Z0-9_+:./\<>;[] | '-' | ']'
> but such a change might break existing substitutions files.

I'd like to agree with this proposal, as we already broke compatibility when 
introducing scopes, and I never understood the reason for being so liberal 
with variable names. In fact, I would further limit them to

  variable-name ::= variable-name-start-char variable-name-char*
  variable-name-start-char ::= [a-zA-Z]
  variable-name-char ::= [a-zA-Z0-9_]

like in most programming languages (LISP being the only exception I am aware 
of). On the other hand, adapting substitution files to the new scoping rules 
is trivial (if tedious, just add the missing definitions), but adapting 
variable names could be quite disruptive; we might want to be careful here.

For reference, I have attached the complete grammar with the changes I 
proposed.

Cheers
Ben

Helmholtz-Zentrum Berlin für Materialien und Energie GmbH   
Hahn-Meitner-Platz 1, 14109 Berlin   
Vorsitzende des Aufsichtsrates: Prof. Dr. Dr. h.c. mult. Joachim Treusch   
Stellvertretende Vorsitzende: Dr. Beatrix Vierkorn-Rudolph Geschäftsführer: Prof. Dr. Anke Rita Kaysser-Pyzalla, Prof. Dr. Dr. h.c. Wolfgang Eberhardt, Dr. Ulrich Breuer   
Sitz der Gesellschaft: Berlin Handelsregister: AG Charlottenburg, 89 HRB 5583   

Disclaimer automatically attached by the E-Mail Security Appliance   
mail0.bessy.de 08/24/10 at Helmholtz-Zentrum Berlin GmbH. 
substitution-file ::= ( global-definitions | template-substitutions )*

global-definitions ::= 'global' '{' variable-definitions '}'
variable-definitions ::= ( variable-definition ','? )*
variable-definition ::= variable-name '=' value

template-substitutions ::= 'file' file-name '{' substitutions '}'
substitutions ::= variable-substitutions | pattern-substitutions
variable-substitutions ::= ( '{' variable-definitions '}' )*

pattern-substitutions ::= 'pattern' '{' pattern-names '}' pattern-definitions
pattern-names ::= ( variable-name ','? )*
pattern-definitions ::= ( '{' pattern-values '}' )*
pattern-values ::= ( value ','? )*

variable-name ::= variable-name-start-char variable-name-char*
file-name ::= file-name-char+ | double-quoted-string | single-quoted-string
value ::= value-char+ | double-quoted-string | single-quoted-string

double-quoted-string ::= '"' (double-quoted-char | escaped-char)* '"'
single-quoted-string ::= "'" (single-quoted-char | escaped-char)* "'"
double-quoted-char ::= [^"\]
single-quoted-char ::= [^'\]
escaped-char ::= '\' .

variable-name-start-char ::= [a-zA-Z]
variable-name-char ::= [a-zA-Z0-9_]
file-name-char ::= [a-zA-Z0-9_+:./\] | '-'
value-char ::= [a-zA-Z0-9_+:./\<>;] | '-' | '[' | ']'

Replies:
Re: msi again Andrew Johnson
References:
msi again Benjamin Franksen
Re: msi again Benjamin Franksen
Re: msi again Andrew Johnson

Navigate by Date:
Prev: Re: msi again Benjamin Franksen
Next: Re: GNU make 3.82 problem Eric Norum
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  <20102011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
Navigate by Thread:
Prev: Re: msi again Andrew Johnson
Next: Re: msi again Andrew Johnson
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  <20102011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
ANJ, 02 Sep 2010 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·