Dylan program structure and macro parsing 
Author Message
 Dylan program structure and macro parsing

I'm trying to puzzle out how Dylan programs are supposed to be parsed, and
I'm sort of confused.

First, though it isn't stated explicitly in the DRM (online PDF version),
it would seem that at least one of a compilation unit's (library's) module
definitions needs to be located within the "dylan-user" module.  For
instance, a module like the following:

        Module: my-module

        define module my-module
                use dylan;
        end;

        // ...whatever...

couldn't be parsed because without knowing the presence of the "use dylan;"
the parser wouldn't know what "define module" meant.  (All of the "define
..." constructs, except for "define macro", are built-in macro definitions
defined in the "dylan" module of the "Dylan" library.)  This would have to
be expressed as:

        Module: dylan-user

        define module my-module
                use dylan;
        end;

        Module: my-module

        // ...whatever...

since the dylan-user module already has a "use dylan" built in.  Come to
think of it, the "define library" definition would have to be in the
dylan-user module too, otherwise the parser wouldn't know what the import
declarations were referring to.

Is this accurate, or am I missing something?  The examples in the Mindy
distribution seem to ignore this restriction.  (Then again, they can, since
Mindy has changed "define module", "define library", etc., from macros to
reserved words built into the grammar.)




Mon, 07 Jun 1999 03:00:00 GMT  
 Dylan program structure and macro parsing


Quote:

> I'm trying to puzzle out how Dylan programs are supposed to be parsed, and
> I'm sort of confused.

> First, though it isn't stated explicitly in the DRM (online PDF version),
> it would seem that at least one of a compilation unit's (library's) module
> definitions needs to be located within the "dylan-user" module....  Come to
> think of it, the "define library" definition would have to be in the
> dylan-user module too, otherwise the parser wouldn't know what the import
> declarations were referring to.

> Is this accurate, or am I missing something?

You are correct.  I'm surprised that this is not covered in the "Program
Structure" chapter of the Dylan Reference Manual.  Maybe we authors
thought it was obvious.

I think it's conventional to put all define library and define module
statements (really definitions) in the Dylan-User module.  About the only
time you wouldn't do this would be if some of these definitions were the
result of macro expansion, and the macro was not available in the
Dylan-User module.  In any case a module's definition can't be inside the
same module, nor inside a module that uses it.

Quote:
> The examples in the Mindy
> distribution seem to ignore this restriction.

What is Mindy an acronym for?  Mindy Is Not Dylan (Yet), right?
--Dave Moon


Mon, 07 Jun 1999 03:00:00 GMT  
 Dylan program structure and macro parsing


Quote:
>Come to
>think of it, the "define library" definition would have to be in the
>dylan-user module too, otherwise the parser wouldn't know what the import
>declarations were referring to.

>Is this accurate, or am I missing something?

You're correct.  The purpose of the dylan-user module is to give you a
foothold to start.  For the simplest portable dylan program, you define
your library and module in one file, in the dylan-user module, and your
program in a second file, in the module you created in the first file.

[self-promotion:  Cf. pp 20-24 of _Dylan_Programming_ for a simple example,
or ch. 13 of the same which is an in-depth discussion of the use of Dylan
modules and libraries.  The simple example is available on-line at
<http://www.harlequin.com/books/dylan-book/index.html> under "Quick Start"]



Mon, 07 Jun 1999 03:00:00 GMT  
 Dylan program structure and macro parsing


Quote:


>>>Come to
>>>think of it, the "define library" definition would have to be in the
>>>dylan-user module too, otherwise the parser wouldn't know what the import
>>>declarations were referring to.

>>>Is this accurate, or am I missing something?

>>You're correct.  The purpose of the dylan-user module is to give you a
>>foothold to start.  For the simplest portable dylan program, you define
>>your library and module in one file, in the dylan-user module, and your
>>program in a second file, in the module you created in the first file.

>This isn't quite right.  A simpler portable dylan program would all be
>in the dylan-user module.  No need to create a second module.  And if
>you're using a file-based system, no need to creat a second file.

Right, although the original post was in terms of creating your own module.
To be scrupulously accurate, I should have said "the simplest Dylan program
in a user-defined module".  Perhaps I should have also said "component"
instead of "program", since "program" may imply "stand-alone program" to
many people.

The very simplest Dylan program, as you describe, could be defined
completely within the dylan-user module; with the limitation that it could
_only_ be a stand-alone program (as opposed to a component of a larger
system, there being no way to add exports to the dylan-user module).
Personally, I would only write a program in the dylan-user module if it
were a one-off, or trial code.



Tue, 08 Jun 1999 03:00:00 GMT  
 Dylan program structure and macro parsing

OK, another question that the online DRM seems to be silent about: does a
macro definition have to lexically precede all macro calls in the same
module?  (It should still be possible to parse programs whose macro calls
come before their definitions, simply by scanning for the token sequence
"define macro" before parsing anything else, but that's less space and time
efficient for the parser.)




Wed, 09 Jun 1999 03:00:00 GMT  
 Dylan program structure and macro parsing

Quote:
> OK, another question that the online DRM seems to be silent about: does a
> macro definition have to lexically precede all macro calls in the same
> module?  (It should still be possible to parse programs whose macro calls
> come before their definitions, simply by scanning for the token sequence
> "define macro" before parsing anything else, but that's less space and
> time
> efficient for the parser.)

Well, I think this is implementation-dependent. A simple compiler might
require that macros lexically proceed macro calls in the same file.
However, another implementation could scan the file for macro definitions
first, then compile the file, as you suggested.

Apple Dylan uses a method similar to the latter: it scans any uncompiled
source for definitions (it doesn't implement macros, but definitions are
the general case), then compiles the code. Apple Dylan is an example of an
implementation that isn't file based.

Ideally, an implementation should not be file-based, at least in the sense
that it can scan and compile single definitions without scanning the entire
file. Then, scanning for definitions before parsing imposes less overhead
because you can compile small bits of source as changes are made, so even
this "long" process can be quite fast, as far as the user is concerned.

...........................................................................
Chris Page                                       define module mail-message
Dylan Hacker                                       use dylan;
Harlequin, Inc.                                    use standard-disclaimer;
<http://www.best.com/~page/>                     end module mail-message;



Wed, 09 Jun 1999 03:00:00 GMT  
 Dylan program structure and macro parsing

Quote:

> > OK, another question that the online DRM seems to be silent about: does a
> > macro definition have to lexically precede all macro calls in the same
> > module?  (It should still be possible to parse programs whose macro calls
> > come before their definitions, simply by scanning for the token sequence
> > "define macro" before parsing anything else, but that's less space and
> > time
> > efficient for the parser.)

In general the DRM doesn't address questions of order of execution.  As I
recall this is because there were some difficult issues that couldn't be
resolved on schedule, so it was decided to remain silent rather than
saying something which might later, after further thought, turn out to be
wrong.

That's probably the real reason the DRM doesn't say anything about modules
having to be defined "before" code in them, the original question in this
thread.

Quote:
> Well, I think this is implementation-dependent. A simple compiler might
> require that macros lexically proceed macro calls in the same file.
> However, another implementation could scan the file for macro definitions
> first, then compile the file, as you suggested.

I don't think it's possible to scan correctly over a call to a macro whose
definition is unknown.  You can't know how to match up the "end" tokens,
and you can't know whether the adjacent tokens "define" "macro" are the
beginning of a macro definition, or just two random tokens in the middle
of a macro call.  Not all macro bodies have the syntax of "body" as
defined in the Dylan grammar.  Although the DRM does not seem to say
anything about this, I think it would be reasonable to assume that a macro
definition must precede any calls to that macro that appear in the same
source record.  I don't think an implementation could be severely
criticized for requiring a macro definition to be in a separate source
record from all calls to the macro.

More generally, any discussion of "files" in the context of the language
specification, rather than any particular implementation, can only be
talking about the Dylan Interchange Format.  Note that in Dylan
Interchange Format, there is one source record per file.

Quote:
> Apple Dylan uses a method similar to the latter: it scans any uncompiled
> source for definitions (it doesn't implement macros, but definitions are
> the general case), then compiles the code. Apple Dylan is an example of an
> implementation that isn't file based.

This is not accurate.  Apple Dylan TR does implement macros, although not
exactly according to the DRM (it implements an older macro
specification).  Apple Dylan TR does not do anything like scanning for the
token sequence "define macro" in the middle of source records.  The
interesting thing about Apple Dylan TR is that it does not require any
ordering between a definition of a variable or constant (including a class
or method) and another definition that uses it.  I forget whether this
freedom extends to macros.  Apple Dylan TR works best when there is only
one definition per source record, which is not typical when using files in
Dylan Interchange Format (remember: there is only one source record per
file).

Quote:
> Ideally, an implementation should not be file-based, at least in the sense
> that it can scan and compile single definitions without scanning the entire
> file. Then, scanning for definitions before parsing imposes less overhead
> because you can compile small bits of source as changes are made, so even
> this "long" process can be quite fast, as far as the user is concerned.

The easiest way to understand how Apple Dylan TR does this is to consider
each source record to be stored in a separate file, with the user relieved
of the responsibility for naming the files, and a special "archiving" file
system used that packs several of these files into one OS file for
efficiency.  The user chooses the file boundaries, i.e. source record
boundaries, but it's customary to put one definition per source record.

I don't believe it is possible automatically to divide a source text into
definitions without parsing the entire source text.  The only way to do it
would be to use heuristics which only work if the source code follows
certain conventions, which I have not seen written down anywhere.
--Dave Moon



Wed, 09 Jun 1999 03:00:00 GMT  
 
 [ 7 post ] 

 Relevant Pages 

1. Dylan program structure and macro parsing

2. MVS Assmbler Structured Programming Macros

3. Dylan Programming Book and Apple Dylan

4. are dylan macros Turing-complete?

5. C++ Templates used like Dylan Macros

6. C++ Templates used like Dylan Macros

7. Simple Reader Macros for Dylan

8. Simple Reader Macros for Dylan

9. dylan parser: Macros

10. Simulating call-by-reference with Dylan macros

11. Simulating call-by-reference with Dylan macros

12. The power of Dylan macro's

 

 
Powered by phpBB® Forum Software