Experimental Physics and
Industrial Control System

Benjamin Franksen <[email protected]> · Wed, 25 Aug 2010 11:12:44 +0200

Hi Andrew

I agree with your comments wrt empty/non-empty pattern substitutions. I am 
glad that you agree with my suggestions in principle (empty lists, optional 
comma, etc). How to exactly formalize the grammar (perhaps so it translates 
to yacc code better) feels like an implementation detail to me; I have no 
objections here.

> > > pattern-substitutions ::=
> > > 
> > >     'pattern' '{' pattern-names '}' pattern-definitions
> 
> I have a question about the above rule: In Yacc I can easily express the
> fact that you can't provide substitution values with no names for them
> thusly:
> 
> pattern_substitutions: PATTERN O_BRACE C_BRACE
> 
>     | PATTERN O_BRACE pattern_names C_BRACE
>     | PATTERN O_BRACE pattern_names C_BRACE pattern_definitions
> 
>     ;
> 
> The following EBNF doesn't convey the same meaning though, it allows
> patterns that the Yacc would reject:
> 
> pattern-substitutions ::=
>     'pattern' '{' pattern-names? '}' pattern-definitions?
>
> The closest I can come up with is this, but it doesn't map onto the Yacc
> code terribly well since it takes 2 rules to Yacc's one:
> 
> pattern-substitutions ::= 'pattern' ( '{' '}' | pattern-block )
> pattern-block ::= '{' pattern-names '}' pattern-definitions?
> 
> Do you have any better ideas?

Since both pattern_names and pattern_definitions cannot be empty, this 
should be equivalent to the yacc rule:

pattern_substitutions: 'pattern' '{' '}'
                     | 'pattern' '{' pattern_names '}' pattern_definitions?

However, such a rule does not allow

  pattern {} {} {}

which I would take to mean "insert the file content twice with no (extra) 
substitutions". This might not look very useful but it could easily appear 
as a corner case when automatically generating pattern definitions, so we 
might want to allow it. So I maintain that

pattern_substitutions: 'pattern' '{' pattern_names? '}' pattern_definitions?

is the better rule. I think it maps to this yacc code:

pattern_substitutions: PATTERN O_BRACE C_BRACE
    | PATTERN O_BRACE C_BRACE pattern_definitions
    | PATTERN O_BRACE pattern_names C_BRACE
    | PATTERN O_BRACE pattern_names C_BRACE pattern_definitions
    ;

> > I'd like to agree with this proposal, as we already broke compatibility
> > when introducing scopes, and I never understood the reason for being so
> > liberal with variable names.
> 
> I have an explanation for that; it reduces the amount of code you have to
> write in the lexer if you only have one definition for what Perl calls a
> Bareword, i.e. something that you're really capturing as a string but you
> don't want users to have to put quotes round.  dbLoadTemplate_lex.l has a
> single token WORD it uses for all Bareword elements, and actually I'm not
> quite sure how to tell it to look for a different kind of Bareword;
> normally information flows from Lex to Yacc but not backwards (it
> probably involves writing start states, which I haven't used before
> myself).

I don't think it is that bad -- fortunetly we don't have to parse C ;-). 
There is no need for lexer states and much less for feeding back parser 
information into the lexer. You just need too different kinds of token, 
let's call them 'bareword' and 'identifier'.

Lexer generators usually try the rules in the order in which they appear in 
the source, so having

identifier {...}
bareword {...}

will give you an identifier token if possible (i.e. identifier takes 
precedence over bareword). This is quite similar to how keywords (reserved 
words) and identifiers are typically handled: you don't want to parse 'if' 
as an identifier, even though it fits the rule.

To parse the value of a macro define a yacc rule that accepts *either* an 
identifier or a bareword or a quoted string, whereas the macro name has to 
be an identifier.

> > In fact, I would further limit them to
> > 
> >   variable-name ::= variable-name-start-char variable-name-char*
> >   variable-name-start-char ::= [a-zA-Z]
> >   variable-name-char ::= [a-zA-Z0-9_]
> 
> I would want to add _ to variable-name-start-char, but otherwise I agree
> with those.  However...
> 
> > like in most programming languages (LISP being the only exception I am
> > aware of). On the other hand, adapting substitution files to the new
> > scoping rules is trivial (if tedious, just add the missing
> > definitions), but adapting variable names could be quite disruptive;
> > we might want to be careful here.
> 
> I think we have to leave the original name definitions in until 3.15 at
> least.

Fine.

> > For reference, I have attached the complete grammar with the changes I
> > proposed.
> 
> Returning mine, which I have working in dbLoadTemplate.y.  The following
> a working replacement for the regular makeBaseApp example template BTW:
> 
> # Example substitutions file
> global { user = anjHost }
> 
> file "db/dbExample1.db" { {} }
> 
> global {
>     scan = "$(no) second"
> }
> file db/dbExample2.db {
>     pattern { no, scan }
>         { 1 }
>         { 2 }
>         { 3, "5 second" }
> }

Very nice. Thanks for your work on this!

Cheers
Ben

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System