macros (was: Re: Say, how about parentheses?) 
Author Message
 macros (was: Re: Say, how about parentheses?)

In-Reply-To: Brian Rogoff's message of Tue, 21 Jan 1997 10:07:33 -0800 (PST)

Quote:
>    No, I haven't written *any* non-trivial programs in Dylan, only
> fairly trivial ones (all less than 2K lines or so) in Mindy. And when I
> did use Lisp, I tended to be fairly wary of macros. In my limited
> experience they are powerful for the writer, and painful for the maintainer
> (or user/adapter) of code. I would be interested in hearing the perceptions
> of others on that topic.

It depends on the macros.  There seems to be a tendency to think
that procedure calls are bound to be easier to understand than
macros, even though you'd have to look at the procedure definition,
or read some documentation, to see what the procedure did -- which
is the same as what you'd have to do for macros.

It's sometimes acknowledged that the people who write macros may
find their own macros easy to understand, but that (rightly) does
nothing to persuade those who are worried about having to deal
with macros written by others.

But consider this:

  (a) That macros have greater potential to confuse (if that's true)
      does not mean that all macros will be more confusing than
      what would be written as an alternative.  Perhaps well-
      designed macros will be less confusing.

  (b) There are some conventions that macros can, and typically
      do, follow.  (Trivial example: that "defining forms" begin
      with "def".)  That means you can tell something about the
      macro right away, before looking up the definition or the
      documentation.

  (c) An expression written with macros may well be much shorter
      and less cluttered than the alternative.

  (d) Many macros correspond very directly to procedure calls.
      For instance, I have a macro that lets you write calls
      like this:

         (with-timeout *time-limit*
           (do-some-stuff))

      which expands into

         (call-with-timeout *time-limit*
           #'(lambda () (do-some-stuff)))

      ("with-" is another of the fairly standard conventions, BTW.)

Unfortunately, some of the pro-macro "propaganda" has turned out
to work _against_ macros in many cases.  In particular, consider
the idea that macros let you define a new language better suited
to your problem and then write in that language, or that macros let
you do the same kind of language extending that the language designers
can.

The first of these, at least, points to a very strong advantage that
macros provide.  But who wants to deal with a new language just to
read, modify, etc a program?  Isn't it hard enough to learn the
base language -- Common Lisp or Scheme or whatever -- without having
to learn _another_ language just to deal with a program?

So this advantage provided by macros -- and it really is an advantage --
can sound like something one would run a mile to avoid.

Indeed, it's not yet clear that macros in Dylan will work as well as
macros in Lisp.  The "not another language!" problem may be worse.
Macros may turn out to be a lot harder to deal with in Dylan than they
are in Common Lisp or Scheme, because of Dylan's syntax.  This goes
back to the old point that Lisp is a "ball of mud": you can add more
mud to it, and it's still a ball of mud.  It may be that macros in
Lisp are less like a new language (that will be a pain to learn, etc)
in Lisp than in Dylan.

Quote:
>                          I am assuming that the more complex infix Dylan
> syntax would make *understanding* largish macros even more difficult than
> Lisp/Scheme; but until there is a Dylan environment for Windows NT
> available which supports the Dylan macro system, it is only speculation.

Dylan has a sort of pattern-language for writing macros that is
bound to make _some_ (even many) macros more readable than they
would be in Lisp, unless a pattern language was also used in Lisp.
(Such things are available and one is standard in Scheme.)

-- jeff



Sun, 11 Jul 1999 03:00:00 GMT  
 macros (was: Re: Say, how about parentheses?)



Quote:
>It depends on the macros.  There seems to be a tendency to think
>that procedure calls are bound to be easier to understand than
>macros, even though you'd have to look at the procedure definition,
>or read some documentation, to see what the procedure did -- which
>is the same as what you'd have to do for macros.

While it's true that you need to look at the function definition or
documentation to determine specific details, e.g. how the return value
relates to the parameters, and it has side effects, at least functions have
some well-defined features, e.g. that all the parameters are evaluated
before being passed to the function.  With macros, you can tell nothing
about the semantics just by looking at the invocation, except perhaps by
assuming that it follows some popular conventions.

So I think it's safe to say that function calls permit someone reading the
code to tell more about the semantics than macros, so they are easier to
understand.

Furthermore, people learning Lisp are taught about functions first, so they
generally have more experience with them.  And writing functions is much
easier than writing macros; writing code is hard, and writing code that
writes code is even harder.
--
Barry Margolin
BBN Planet, Cambridge, MA

(BBN customers, please call (800) 632-7638 option 1 for support)



Mon, 12 Jul 1999 03:00:00 GMT  
 macros (was: Re: Say, how about parentheses?)



Quote:
> It depends on the macros.  There seems to be a tendency to think
> that procedure calls are bound to be easier to understand than
> macros, even though you'd have to look at the procedure definition,
> or read some documentation, to see what the procedure did -- which
> is the same as what you'd have to do for macros.

Yep, documention is critical. Forth programmers solve this
problem by documention their extentions to the language. I'm
sure that this should also work for Lisp.

I think that the purpose of a macro should be documented as
well as its behaviour. The _intention_ should be clear.
Otherwise, a macro could easily obscure the meaning of the
code.

Quote:
> It's sometimes acknowledged that the people who write macros may
> find their own macros easy to understand, but that (rightly) does
> nothing to persuade those who are worried about having to deal
> with macros written by others.

Macro style is also important. See below.

Quote:
> But consider this:

>   (a) That macros have greater potential to confuse (if that's true)
>       does not mean that all macros will be more confusing than
>       what would be written as an alternative.  Perhaps well-
>       designed macros will be less confusing.

W&H talk about "programming cliches", and using macros to
encapsulate them into a single, and hopefully simple to use,
construct. Designing such things well is not easy. I suspect
that some of us are better at it than others. A good Forth
or Lisp programmer will be good at abstracting symantics into
new language constructs.

Quote:
>   (b) There are some conventions that macros can, and typically
>       do, follow.  (Trivial example: that "defining forms" begin
>       with "def".)  That means you can tell something about the
>       macro right away, before looking up the definition or the
>       documentation.

Agreed. In CL, there are conventions for macro behaviour.
User defined macros should conform, to avoid confusion.
CL has a lot more macros than Scheme, so this is perhaps
more important is CL than Scheme. Still, if you're a Scheme
programmer who uses a lot of macros in your code, then this
issue will be no less important.

I'd recommend documenting any additional conventions you
use in your macros, plus intention etc.

Quote:
>   (c) An expression written with macros may well be much shorter
>       and less cluttered than the alternative.

Too right! If you repeat a "pattern" just a few times, that
may be a good enough reason to encapsulate it in a macro.
I'v often written a macro that was only used _once_ in a
program, and yet I still felt that it made the code clearer.
This is due to the conventions that already exist.

The same is true for higher-order functions.

Quote:
>   (d) Many macros correspond very directly to procedure calls.
>       For instance, I have a macro that lets you write calls
>       like this:

>          (with-timeout *time-limit*
>            (do-some-stuff))

>       which expands into

>          (call-with-timeout *time-limit*
>            #'(lambda () (do-some-stuff)))

>       ("with-" is another of the fairly standard conventions, BTW.)

The "with-" convention in Scheme works with functions as well
as it works with macros in CL. I love being able to build on
such conventions, but they're not unique to Lisp. In fact, I
first encountered this idea in Smalltalk. Now I'm finding it
in more recent languages like Haskell.

Quote:
> Unfortunately, some of the pro-macro "propaganda" has turned out
> to work _against_ macros in many cases.  In particular, consider
> the idea that macros let you define a new language better suited
> to your problem and then write in that language, or that macros let
> you do the same kind of language extending that the language designers
> can.

> The first of these, at least, points to a very strong advantage that
> macros provide.  But who wants to deal with a new language just to
> read, modify, etc a program?  Isn't it hard enough to learn the
> base language -- Common Lisp or Scheme or whatever -- without having
> to learn _another_ language just to deal with a program?

The documentation could include a tutorial. As I've said,
this problem isn't unique to Lisp - it's specific to all
programs that are badly documented!

Quote:
> So this advantage provided by macros -- and it really is an advantage --
> can sound like something one would run a mile to avoid.

If poorly used, yes. The same is true for _anything_.
Perhaps Lisp suffers more from this than C++, but I
don't know. I never have trouble reading my own code,
but dealing with other peoples' code can often be agony.
I find loads of assumptions about what I know, and I
sometimes find these are things that I _don't_ know,
and so get lost in the code.

Even when I can understand what I'm looking at, it takes
time to achieve it. Also, if you don't understand what
the code actually _does_, and why, then understanding how
it does it won't help you much. A simple hint, like a
reference to a CS subject (or even book!) may help.

For example, I wouldn't bother writing an explanation of
a garbage collector in my code - I'd simply refer the
reader to the source of the GC algorithm, and note the
changes I've made.

Quote:
> Indeed, it's not yet clear that macros in Dylan will work as well as
> macros in Lisp.  The "not another language!" problem may be worse.
> Macros may turn out to be a lot harder to deal with in Dylan than they
> are in Common Lisp or Scheme, because of Dylan's syntax.  This goes
> back to the old point that Lisp is a "ball of mud": you can add more
> mud to it, and it's still a ball of mud.  It may be that macros in
> Lisp are less like a new language (that will be a pain to learn, etc)
> in Lisp than in Dylan.

Macros can also obscure C/C++ code, if over used. It's always
important to document these things, and ensure that whoever
reads the source will find the docs. A hint in a comment, in
the source code itself, might help. I often write macros (in
C/C++ and Lisp) which have a limitd scope - not just a single
file, but a very small part of that source file. Dispite advise
to put macros in a special macro file, I tend to put these
special cases immediately before the code that used them.
Even without documentation, this implies that the macro
has limited "scope", in terms of usuage.

It's unfortunate that Lisp doesn't make it easy to enforce
scoping of macro names, but it is possible. In CL, you could
use macrolet, or use a symbol local to a package. I don't
know how this might be done in Scheme, as it's beyond the
scope (no pub intended) of the language.

Dylan handles this much better.

Quote:
> Dylan has a sort of pattern-language for writing macros that is
> bound to make _some_ (even many) macros more readable than they
> would be in Lisp, unless a pattern language was also used in Lisp.
> (Such things are available and one is standard in Scheme.)

I've not yet found a Scheme that uses a hygenic macro system.
However, I can see how one could be added to a Lisp that
uses a less sophisticated system, like CL. That way you have
a choice: you can write declarative macros most of the time,
but also write macros that have side effects when you feel
you need them. I'v e always found it easier to write macros
where the expansion creates the side effect, which is why
I'd prefer the declarative style if it were available.
--
<URL:http://www.wildcard.demon.co.uk/> You can never browse enough
  Martin Rodgers Y Developer and Information Broker Y London, UK
       Please remove the "nospam" if you want to email me.
                  "Blow out the candles, HAL."


Wed, 14 Jul 1999 03:00:00 GMT  
 macros (was: Re: Say, how about parentheses?)


Quote:

> So I think it's safe to say that function calls permit someone reading the
> code to tell more about the semantics than macros, so they are easier to
> understand.

Macros are one of those power-user features. =) Used correctly, they can
make a large program much more readable and maintainable. Used poorly, they
usually make an incomprehensible mess.

For example, a program might open and close many files:

  let myfile = open-file("/usr/local/mylog.txt", direction: #"output",
                         mode: #"append");
  block ()
    format(myfile, "Did something.\n");
    // Write some other stuff...
  cleanup
    close-file(myfile);
  end block;

Fooling around with blocks and cleanup clauses can become tedious quickly.
A short macro would make life easier (this is based on the standard streams
library):

  define macro with-open-file
    { with-open-file(?stream:name = ?file:expression,
      #rest ?options:*) ?:body end }
    => { let ?stream = open-file(?file, ?options);
         block ()
           ?body
         cleanup
           close-file(stream)
         end }
  end macro;

  with-open-file (myfile = "/usr/local/mylog.txt", direction: #"output",
                  mode: #"append")
    format(myfile, "Did something.\n");
    // Write some other stuff...
  end with-open-file;

Macros are just another abstraction technique. Like functions and
constants, they allow a programmer to package up some idea and refer to it
by name. Instead of representing values or steps in a computation, they
represent some more complicated structure of code.

Quote:
> Furthermore, people learning Lisp are taught about functions first, so they
> generally have more experience with them.  And writing functions is much
> easier than writing macros; writing code is hard, and writing code that
> writes code is even harder.

Definitely. Macros are a powerful tool for library designers and
"toolsmiths" on large projects. Ordinary programmers may create simple,
local macros, but usually seem to shy away from writing big hairballs such
as 'define class' or 'for'.

Cheers,
Eric

http://www.pobox.com/~emk/



Thu, 15 Jul 1999 03:00:00 GMT  
 macros (was: Re: Say, how about parentheses?)


Quote:

> In-Reply-To: Brian Rogoff's message of Tue, 21 Jan 1997 10:07:33 -0800 (PST)

> >       No, I haven't written *any* non-trivial programs in Dylan, only
> > fairly trivial ones (all less than 2K lines or so) in Mindy. And when I
> > did use Lisp, I tended to be fairly wary of macros. In my limited
> > experience they are powerful for the writer, and painful for the maintainer
> > (or user/adapter) of code. I would be interested in hearing the perceptions
> > of others on that topic.

> It depends on the macros.  There seems to be a tendency to think
> that procedure calls are bound to be easier to understand than
> macros, even though you'd have to look at the procedure definition,
> or read some documentation, to see what the procedure did -- which
> is the same as what you'd have to do for macros.

        IMO, short macros are about as easy to understand as short
procedures, i.e. very easy, but as size grows macro understanding
complexity grows much faster. Perhaps I never fully grokked Common
Lisp style macros (thats what I have experience with), and perhaps
Scheme or some as yet undesigned macro system doesn't have this property.
Note that I am not arguing for the omission of macros, a very powerful
feature I know, just extreme care in their use, especially in long lived
programs which will be maintained by many programmers.

Quote:

> It's sometimes acknowledged that the people who write macros may
> find their own macros easy to understand, but that (rightly) does
> nothing to persuade those who are worried about having to deal
> with macros written by others.

> But consider this:

>   (a) That macros have greater potential to confuse (if that's true)
>       does not mean that all macros will be more confusing than
>       what would be written as an alternative.  Perhaps well-
>       designed macros will be less confusing.

        Absolutely. I've seen lots of pre-processors used to make C into
"something else". Understanding Lisp macros is usually far easier.

<...lots of stuff I agree with elided...>

Quote:
> Unfortunately, some of the pro-macro "propaganda" has turned out
> to work _against_ macros in many cases.  In particular, consider
> the idea that macros let you define a new language better suited
> to your problem and then write in that language, or that macros let
> you do the same kind of language extending that the language designers
> can.

        This is a good example. I have no problem with this kind of
language design experimentation, but the potential for creating
unmaintainable messes is large. It is probably just a matter of taste,
but I prefer to use them for simple notational conveniences, and leave
bigger stuff (like implementing the object system in straight Lisp) to
the compiler. My brain is just too small.

Quote:
> Indeed, it's not yet clear that macros in Dylan will work as well as
> macros in Lisp.  The "not another language!" problem may be worse.
> Macros may turn out to be a lot harder to deal with in Dylan than they
> are in Common Lisp or Scheme, because of Dylan's syntax.  

        This is what I suspect. Lisp (here I really mean parenthesized
prefix syntax) is syntactically trivial, so source to source
transformations on this representation of code are easier to understand
than on an Algolesque syntax. Of course, the current Dylan macro system is
not like the CL macro system, so this probably has compensating advantages
(easier to understand) and disadvantages (less powerful).

Quote:
> This goes back to the old point that Lisp is a "ball of mud": you can
> add more mud to it, and it's still a ball of mud.  It may be that macros in
> Lisp are less like a new language (that will be a pain to learn, etc)
> in Lisp than in Dylan.

> >                          I am assuming that the more complex infix Dylan
> > syntax would make *understanding* largish macros even more difficult than
> > Lisp/Scheme; but until there is a Dylan environment for Windows NT
> > available which supports the Dylan macro system, it is only speculation.

> Dylan has a sort of pattern-language for writing macros that is
> bound to make _some_ (even many) macros more readable than they
> would be in Lisp, unless a pattern language was also used in Lisp.
> (Such things are available and one is standard in Scheme.)

        Right, the simplified macro system will help, but I think the infix
syntax hurts, for macros anyways. This said, I am still looking forward to
the arrival of some "real" Dylan environments, since it seems like it will
be a very pleasant language for many applications.

-- Brian



Thu, 15 Jul 1999 03:00:00 GMT  
 macros (was: Re: Say, how about parentheses?)


Quote:

>         IMO, short macros are about as easy to understand as short
> procedures, i.e. very easy, but as size grows macro understanding
> complexity grows much faster. Perhaps I never fully grokked Common
> Lisp style macros (thats what I have experience with), and perhaps
> Scheme or some as yet undesigned macro system doesn't have this property.
> Note that I am not arguing for the omission of macros, a very powerful
> feature I know, just extreme care in their use, especially in long lived
> programs which will be maintained by many programmers.

Macros may need to be understood in two different contexts:
 1) The definition of the macro
 2) The use of the macro

Let's consider small Dylan macros. The definition of a small macro is
fairly easy to understand. Consider the standard macro "unless":

  define macro unless
     { unless (?test:expression) ?:body end }
       => { if  (~?test) ?body end }
  end macro unless;

The use of a small macro is also quite comprehensible, provided the macro
is well-designed. It's possible to create syntactic monstrosities with
comprehensible definitions but incomprehensible invocations, however. Catch
these in code review, just like any other style problem. Small Dylan macros
should be easy to maintain, provided they are used with some minimal degree
of intelligence.

Large Dylan macros are a different beast entirely. They're very hard to
write, and the definitions are nearly incomprehensible. (See the sample
"for" macro in the DRM, pp. 167-171. Scary, huh?) It's often hard just to
decide what an invocation should look like. To be fair, big macros are sort
of like perl programs: they look like random noise until you become a
specialist, at which point they suddenly make sense. It takes a while to
reach that point, though.

Interestingly, a well-designed big macro can be easy to invoke. I find that
the greater variation of syntactic forms in Dylan makes macro invocations
easier to understand than in LISP, where you have a pile of nested
parentheses.

  for (item in collection)
    // ...
  end for;

  for (i from 1 to 10)
    // ...
  end for;

  // Print a LISP-style list (head=car, tail=cdr)
  // Expands into a tail-recursive local method
  for (mylist = #(1, 2, 3) then mylist.tail, until: empty?(mylist))
    format(*standard-output*, "Item: %=\n", mylist.head);
  end for;

A poorly-designed big macro is an utter nightmare, of course.

The decision process for using a small macro might go something like this:

  1) Is it messy to solve this problem without a macro?
  2) Would a macro save keystrokes and make code more readable?
  3) Can I define this macro in under 30 seconds?
  4) Would the macro represent a coherent abstraction?

A good example might be a dozen related function definitions, each
differing in some small way. Using a simple local macro would remove lots
of redundant code, and help express the idea that the functions all belong
to one group:

  define constant $table = /* something */;
  define constant BIT1 = #x00000001;
  // define some more bits

  define macro lookup-function-definer
    { define lookup-function ?:name (?maskbits:*) }
      => { define inline method ?name (char :: <character>)
            => (matches? :: <boolean>)
             logand($table[char], logior(?maskbits)) ~= 0;
           end method; }
  end macro;

  define lookup-function foo? (BIT1, BIT2, BIT3);
  define lookup-function bar? (BIT1, BIT5);
  // ... about 20 more of these predicates

If you're reading the code, you have to stop to understand the macro
definition, but it's really easy to understand all 22 specialized lookup
functions, and verify that they're using the correct bits.

The decision process for using a *big* macro in a production environment
would be quite different:

  1) Will our project, or the clients of our library, frequently
     require complicated and redudant code to express some abstract
     concept?
  2) Is there no tidy way to avoid this by redesigning something?
  3) Would we benefit from a simple, standardized notation for
     representing this abstract concept?
  4) Are we willing to document this notation, and teach it to
     everyone who works on the project?
  5) Are we willing to invest the time to design and refine this
     notation to be compatible with the general concepts and
     conventions of the language?
  6) Are we willing to write 1 to 5 pages of esoteric, specialized
     code to implement our abstraction correctly?

A good example might be the ability to define windows easily. A naive first
draft of the macro might be used as shown below; later revisions would
eliminate some rather subtle extensibility problems and design flaws, which
I won't go into. However, the polished and documented version of this macro
would be quite an asset to anyone building a GUI.

  define window login-window (initially-visible?: #f)
    vertical group {
      string-pane (text: "Enter name: ");
      input-pane (id: #"login", text: *default-user-name*);
      string-pane (text: "Enter password: ");
      input-pane (id: #"password");
      horizontal group {
        button (name: "Cancel", message: $hit-cancel);
        button (name: "OK", message: $hit-ok);
      }
    }
  end window;

Someone from Harlequin posted a far superior implementation of this idea
several months ago. It was extensible, and looked considerably more like
Dylan than the above drek.

Cheers,
Eric

http://www.pobox.com/~emk/



Fri, 16 Jul 1999 03:00:00 GMT  
 macros (was: Re: Say, how about parentheses?)

In-Reply-To: Eric M. Kidd's message of Mon, 27 Jan 1997 17:00:07 -0500

Eric's message is excellent, full of good advice.  However, I'd like
to add something about this:

Quote:
> Large Dylan macros are a different beast entirely. They're very hard to
> write, and the definitions are nearly incomprehensible. (See the sample
> "for" macro in the DRM, pp. 167-171. Scary, huh?) It's often hard just to
> decide what an invocation should look like. To be fair, big macros are sort
> of like perl programs: they look like random noise until you become a
> specialist, at which point they suddenly make sense. It takes a while to
> reach that point, though.

Dylan macros can be very different from Common Lisp macros, because
Dylan has (only) a high-level pattern-language for writing macros,
while Common Lisp macros are defined in Common Lisp.  For some kinds
of complex maros, a definition in ordinary Lisp may be easier to
understand than one in a pattern language.

But just think of how much more experience we have in writing
code in "ordinary" language, and how much less we have when it
comes to the Dylan macro language.  It may be that Dylan programmers
will be able to develop ways of writing complex macros that are
easier to understand.

-- jeff



Sat, 17 Jul 1999 03:00:00 GMT  
 
 [ 7 post ] 

 Relevant Pages 

1. Say, how about parentheses?

2. Say, how about parentheses?

3. less parentheses --> fewer parentheses

4. less parentheses --> fewer parentheses

5. Dangling Closing Parentheses vs. Stacked Closing Parentheses

6. say/pull combo leaving cursor after say?

7. I say 2 you say 1

8. A macro involving two sub-macros - where the 2nd macro needs results from the first

9. I am not deaf, but am I mute?

10. macro -vs- macro/codeblock

11. Help with macros writing macros in Bigloo

12. syntax-rules macros with sub-macros

 

 
Powered by phpBB® Forum Software