Clipper lex & yacc build 6 
Author Message
 Clipper lex & yacc build 6

There is a new build (6) of the Clipper free parser (soon to become a
compiler :-) available from:

ftp://ftp.iag.net/pub/clipper/general/cliparse.zip

Today we were able to parse a Clipper source code with 2720 lines (our
window.ppo). There was just one line giving error: EXIT from inside a IF ...
ENDIF (EXIT is actually defined into a DO WHILE ... loop and it has to
become a statement). Also #line is not supported as it should be managed by
the preprocessor.

Antonio Linares
www.fivetech.com



Mon, 10 Sep 2001 03:00:00 GMT  
 Clipper lex & yacc build 6


Quote:
>There is a new build (6) of the Clipper free parser (soon to become a
>compiler :-) available from:

>ftp://ftp.iag.net/pub/clipper/general/cliparse.zip

  I have some modification for the yacc rules. Note however that I am not
a yacc guru so I can be wrong.

The first modification is the operators precedence rules:

/*the lowest precedence*/
/*postincrement and postdecrement*/
%left  POST
/*assigment - from right to left*/
%right INASSIGN
%left  PLUSEQ MINUSEQ
%left  MULTEQ DIVEQ MODEQ
%left  EXPEQ
/*logical operators*/
%left  OR
%left  AND
%left  NOT
/*relational operators*/
%left  '<' '>' EQ NE1 NE2 LE GE '$' '='
/*mathematical operators*/
%left  '+' '-'
%left  '*' '/' '%'
%left  POWER
%left  UNARY
/*preincrement and predecrement*/
%left  PRE
/*special operators*/

%RIGHT '\n' ';'
/*the highest precedence*/

/*the above precedence rules doesn't distinguish between pre- and post-
operators then the operators precedence have to be moved to the grammar
rules -these rules are referencing to automatically created tokens PRE,
POST, and UNARY -I am not sure if this is correct*/

VarUnary   : Variable INC %prec POST
           | INC Variable %prec PRE
           | Variable DEC %prec POST
           | DEC Variable %prec PRE
           ;

Operators  : Expression '+'    Expression
/* some code snipped */
           | '-' Expression %prec UNARY
           | '+' Expression %prec UNARY
           | VarAssign
           ;

The assignment needs more attention. The following:
VarAssign  : Variable '=' Expression;
is not enough because it doesn't distinguish between the assign context
and the equality context.

example 1:
v1:=1
v2:=2
? v1 = v2
? v1            //this prints: 1

example 2:
v1:=1
v2:=2
v1 = v2
? v1            //this prints: 2

The function calls can use the reference operator. The list of arguments
cannot be defined by 'ExpList' because it is also used in other places
(like 'ArrayIndex') where the reference operator is not allowed. This is
why we need the separate definition of function argument list.

FunCall    : IDENTIFIER '(' ArgList ')'
           ;

ArgList    : /*empty list*/
           | Argument
           | ArgList ',' Argument

Argument   : Expression

           ;

The alias expression is more complicated also:

NotNilExpr : LITERAL
           | Variable
           | VarUnary
           | Operators
           | FunCall
           | IfInline
           | Array
           | ObjectMethod
           | '&' Variable
           ;

AliasExp   : IDENTIFIER ALIAS IDENTIFIER
           | IDENTIFIER ALIAS '(' NotNilExpr ')'
           | '(' NotNilExpr ')' ALIAS '(' NotNilExpr ')'
           ;

Also in "clipper.l" there is no definition for power operator
"**"|"^"    return POWER
Note also that the '**' operator should be declared as the start of a
single line comment.

Regards, Ryszard



Tue, 11 Sep 2001 03:00:00 GMT  
 Clipper lex & yacc build 6
Ryszard,

Excellent! I am going to implement and test your suggestions and I will let
you know about the results.

You look like a yacc real guru !  Soon I will be publishing the preprocessor
lex & yacc rules and I would really appreciate your help again for this
public OpenClipper project.

Thanks so much !

Antonio Linares
www.fivetech.com



Tue, 11 Sep 2001 03:00:00 GMT  
 Clipper lex & yacc build 6
Ryszard,

Quote:
> The first modification is the operators precedence rules:

Very good! I have implemented them and works great. I do like very much the
%prec use you do. So nice!

Quote:
> The assignment needs more attention.

I understand what you say. What do you think ? How could we make the
distinction ?

Quote:
> The function calls can use the reference operator.


single char. Works great!

Quote:
>           | '&' Variable

I see you are considering this macro implementation I wasn't, so I have
created a macro rule:

Macro      : '&' Variable
           | '&' '(' Expression ')'
           ;

Quote:
> The alias expression is more complicated also:

I see you want to avoid there the use for NIL, but I am not sure as we may
use it to report an error  otherwise we may not be able to detect it. I am
thinking about it.

Anyhow, this rule generates a shift/reduce conflict. I am not sure why:

AliasExp   : IDENTIFIER ALIAS IDENTIFIER
                  | IDENTIFIER ALIAS '(' Expression ')'
                  | '(' Expression ')' ALIAS '(' Expression ')'
                  ;

What do you think ? How could we avoid that conflict ?

Quote:
>Also in "clipper.l" there is no definition for power operator
>"**"|"^"    return POWER

Done! Thanks!

Quote:
> Note also that the '**' operator should be declared as the start of a
> single line comment.

I am thinking how to make that distinction from lex. I could detect the
start of a line, but the comment may be somewhere in the line also.

Thanks so much. Your help is beeing really valuable for this OpenClipper (or
whatever name we may finally use) project!

regards,

Antonio Linares
www.fivetech.com



Tue, 11 Sep 2001 03:00:00 GMT  
 Clipper lex & yacc build 6


Quote:
>> The assignment needs more attention.

>I understand what you say. What do you think ? How could we make the
>distinction ?

  I have resolved it: the assignment can be used only as a statement -
every use of '=' in an expression have to be interpreted as a compare
operation.
Then we have to define:

Statement  : ExecFlow Crlf
           | FunCall Crlf
           | ObjectMethod Crlf
           | VarUnary Crlf
           | VarAssign Crlf
           | RETURN Crlf
           | RETURN Expression Crlf
           | PUBLIC VarList Crlf
           | EXITLOOP Crlf
           | LOOP Crlf
           | Variable '=' Expression Crlf { printf("\n= assign\n"); }
           ;

/*VarAssign should be changed*/
VarAssign  : Variable INASSIGN Expression
           | Variable PLUSEQ   Expression
           | Variable MINUSEQ  Expression
           | Variable MULTEQ   Expression
           | Variable DIVEQ    Expression
           | Variable EXPEQ    Expression

/*and the Operators too*/
Operators  : Expression '='    Expression { printf("\n= compare\n"); }
           | Expression '+'    Expression
           /* other rules */
           ;

[....]

Quote:
>> The alias expression is more complicated also:

>I see you want to avoid there the use for NIL, but I am not sure as we may
>use it to report an error  otherwise we may not be able to detect it. I am
>thinking about it.

  Yes, you are right. This is the task for a runtime engine.

Quote:
>Anyhow, this rule generates a shift/reduce conflict. I am not sure why:

>AliasExp   : IDENTIFIER ALIAS IDENTIFIER
>                  | IDENTIFIER ALIAS '(' Expression ')'
>                  | '(' Expression ')' ALIAS '(' Expression ')'
>                  ;

>What do you think ? How could we avoid that conflict ?

  We have to define the alias statement and the alias expression:

AliasStmnt : IDENTIFIER ALIAS '(' ExpList ')'
           | '(' ExpList ')' ALIAS '(' ExpList ')'
           ;

AliasExp   : IDENTIFIER ALIAS IDENTIFIER
           | '(' ExpList ')' ALIAS IDENTIFIER
           | AliasStmnt
           ;

The 'AliasStmnt' should be added to 'Statement' rule

Statement  : ExecFlow Crlf
           | FunCall Crlf
           | ObjectMethod Crlf
           | VarUnary Crlf
           | VarAssign Crlf
           | RETURN Crlf
           | RETURN Expression Crlf
           | PUBLIC VarList Crlf
           | EXITLOOP Crlf
           | LOOP Crlf
           | Variable '=' Expression Crlf { printf("\n= assign\n"); }
           | AliasStmnt Crlf
           ;

In CA-Clipper these are illegal:
alias->(, expr)
alias->( expr,)
v :=arr[ 1, ]
v :=arr[ , ]
{|| expr ,}

This is why we need to change the 'ExpList' and 'Array' rules

Array      : '{' ElemList '}'
           ;

ElemList   : /*empty array*/
           | Expression
           | Expression ',' ElemList
           | ',' ElemList
           ;

/*we need at least one item in the expression list*/
ExpList    : Expression
           | ExpList ',' Expression
           ;

and also the codeblock:

CodeBlock  : '{' '|' '|' ExpList '}'
           | '{' '|' ParamList '|' ExpList '}'
           ;

Finally (for today) we have to define the PROCEDURE token:

Source     : Crlf
           | Include Crlf
           | Extern Crlf
           | StaticsDef Crlf
           | Function
           | Procedure
           | Source Crlf
           | Source Include
           | Source Extern
           | Source StaticsDef
           | Source Function
           | Source Procedure

Procedure  : ProcHead Statements
           | ProcHead VarDefs Statements
           | FunScope ProcHead Statements
           | FunScope ProcHead VarDefs Statements
           ;

ProcHead   : PROCEDURE IDENTIFIER Crlf
           | PROCEDURE IDENTIFIER '(' ')' Crlf
           | PROCEDURE IDENTIFIER '(' ParamList ')' Crlf
           ;

And now the problem:
If the last line of the source is not terminated by CR/LF then this line
causes the 'syntax error' even if it contains a valid expression.

Regards, Ryszard



Wed, 12 Sep 2001 03:00:00 GMT  
 Clipper lex & yacc build 6

Quote:

> And now the problem:
> If the last line of the source is not terminated by CR/LF then this line
> causes the 'syntax error' even if it contains a valid expression.

I think the trick here is to write your own source reader and pass the lines
to yyparse() one at a time after fixing up errors like the above.

Would that work?

--
Take a look in Hagbard's World: |   w3ng - The WWW Norton Guide reader.
http://www.acemake.com/hagbard/ |     eg - Norton Guide reader for Linux.
http://www.hagbard.demon.co.uk/ |    weg - Norton Guide reader for Windows.
Free software, including........| dgscan - DGROUP scanner for Clipper.



Thu, 13 Sep 2001 03:00:00 GMT  
 Clipper lex & yacc build 6

Quote:

> I am thinking how to make that distinction from lex. I could detect the
> start of a line, but the comment may be somewhere in the line also.

In Clipper 5+ it has to be in a new line (leading spaces allowed).
Normal RegExp as I know it
        ^[      ].*\*
(Space and a tab)
Maybe \t for a tab can be used. Procmail doesn't like it.

--
bye
    ranf



Thu, 13 Sep 2001 03:00:00 GMT  
 Clipper lex & yacc build 6
Ryszard,

I want to publically thank you for your great help to this public project.

regards,

Antonio Linares
www.fivetech.com



Fri, 14 Sep 2001 03:00:00 GMT  
 Clipper lex & yacc build 6
Dave,

Quote:
> I think the trick here is to write your own source reader and pass the
lines
> to yyparse() one at a time after fixing up errors like the above.

There is a way to make lex read an entire source code line one at a time. In
fact we are going to use it once we may have the preprocessor working, as we
need to supply it the line to get the possible translations.

We will also use it to show the source code line when getting a syntax
error. This makes the trick:

\n.*            iLine++; iTokenPos = 0; strcpy( linebuf, yytext + 1 );
yyless( 1 ); return '\n';

regards,

Antonio Linares
www.fivetech.com



Fri, 14 Sep 2001 03:00:00 GMT  
 Clipper lex & yacc build 6


Quote:
>> If the last line of the source is not terminated by CR/LF then this line
>> causes the 'syntax error' even if it contains a valid expression.

>I think the trick here is to write your own source reader and pass the lines
>to yyparse() one at a time after fixing up errors like the above.

  I will try it. I think we will have to change the source reader because
it also hangs if the source contains characters with ASCIII > 128.

Regards, Ryszard



Fri, 14 Sep 2001 03:00:00 GMT  
 Clipper lex & yacc build 6


Quote:


>>> If the last line of the source is not terminated by CR/LF then this line
>>> causes the 'syntax error' even if it contains a valid expression.

>>I think the trick here is to write your own source reader and pass the lines
>>to yyparse() one at a time after fixing up errors like the above.

>  I will try it. I think we will have to change the source reader because
>it also hangs if the source contains characters with ASCIII > 128.

You must instructs to lex to generate an 8 bits scanner, in Unix/linux
it's maked with -8 option. In lex for dos I uknown how it is.

lex -i -8 clipper.l

Manuel Ruiz Tienza.



Fri, 14 Sep 2001 03:00:00 GMT  
 Clipper lex & yacc build 6
Ralf,

Many thanks. I have added a new lex rule:

^{SpaceTab}*(\*|"NOTE") BEGIN COMMENT2;
<COMMENT2>.*   BEGIN 0;
<COMMENT2>\n   printf( "\rline: %i", ++iLine ); BEGIN 0;

I have added "NOTE" support which it is a "*" synonym, though it is old
style.
It is working!  :-)

regards,

Antonio Linares
www.fivetech.com



Fri, 14 Sep 2001 03:00:00 GMT  
 Clipper lex & yacc build 6


Quote:
>You must instructs to lex to generate an 8 bits scanner, in Unix/linux
>it's maked with -8 option. In lex for dos I uknown how it is.

>lex -i -8 clipper.l

  Thanks. It worked also for DOS version.

Regards, Ryszard



Fri, 14 Sep 2001 03:00:00 GMT  
 
 [ 13 post ] 

 Relevant Pages 

1. Clipper lex & yacc and parser, build 8

2. Clipper lex & yacc

3. An Ada Translator - <BUILDS, SMUDGE, Lex, and Yacc

4. Q: Lex&YACC smalltalk version ?

5. Lex & Yacc for Smalltalk

6. Harbour Project | Lex & Yacc Guide

7. LEX & YACC

8. (YACC&LEX) PL/I Grammar

9. (YACC&LEX) PL/1 Grammar

10. lex & yacc questions

11. YACC & LEX

12. lex & yacc for scheme

 

 
Powered by phpBB® Forum Software