Defining words, search order and wordlists. 
Author Message
 Defining words, search order and wordlists.

I'm not sure whether doing the following is legal ANS FORTH so I was
looking for someone to provide an interpretation as to whether it is
legal and if not why not.

Here is my problem.  Define the following:

  \ Add selx to search order and make new definitions go into selx.
: set-current-wordlist  ( selx -- )
  >r get-order 1+ r> swap set-order definitions ;

  \ Fill wordlist with default definitions.
: fill-wordlist  ( n n n -- )
  s" constant a" evaluate
  s" constant b" evaluate
  s" constant c" evaluate ;

  \ Create new wordlist with default definitions.
: create-wordlist  ( -- )
  create
    wordlist set-current-wordlist
    1 2 3 fill-wordlist
  does>            ( body -- )
    set-current-wordlist ;

  \ Create new wordlist with default definitions, revised.
: create-wl  ( -- )
  wordlist dup set-current-wordlist
  1 2 3 fill-wordlist
  previous definitions
  create
    dup , set-current-wordlist
  does>      ( body -- )

I believe that the above words set-current-wordlist and fill-wordlist by
themselves conform to the standard.  My problem lies with the word
create-wordlist.  Create-wordlist tries to create a defining word which
creates a wordlist, adds the newly created wordlist to the search order,
set the current definitions to go into the newly created wordlist, then
fills the wordlist with some common definitions.

When the word create-wordlist is executed (ie. create-wordlist abc) it
produces a new (ie. abc) which PFE, an ANS FORTH, is marked in the
dictionary as a variable -- not a defining word and c in the newly
created wordlist is marked in its dictionary as a defining word -- not
a constant!  As far as I can tell from the standard, this could be a
result of does> (section 6.1.1250) replacing the execution semantics of
the most recent definition, ie. abc which was defined by create, and the
fact that fill-wordlist creates new definitions.  It seems that section
"3.4.5 Compilation" of the standard may apply somehow as well.

The standard seems unclear to me when it says that: "does> replaces the
execution semantics of the most recent definition."  I would have expected
it to use the term "current definition" from "2.1 Definitions of terms."
But again whether its "recent definition" or "current definition" it really
says to me: "replaces the execution semantics of the most recent/current
definition [in the most recent/current definition's wordlist]".  Note I
added the bracketed item to show what the standard implies to me.  However
it is clear that PFE, an ANS FORTH, implies "in the current wordlist where
definitions are being placed."

It's unclear in the standard, to me, which one is correct.  Given the
ambiguity I rewrote the word create-wordlist and it appears as the word
create-wl.  The new word create-wl now creates the wordlist fills it with
the common definitions and restores the previous wordlist.  Then create-wl

*creates* the defining word.  This seems to work for PFE but may not work
for other ANS implementations.  However, this points out another possible
problem with the definitions of create-wordlist and create-wl.

The newly preceived problem is that both words change the search order before
the does> is done.  It seems to me that sections 16.3.3 and A.16.3.3 say that
this produces an ambiguous condition.  Is this correct?  Does 16.3.3 refer
to when the word create-wordlist is being compiled or the word being defined
by create-wordlist?

My final question is: "Is it possible to portably create a defining word
which creates a wordlist, fills that wordlist with definitions and then
switches the search order to that wordlist with the current ANS standard?"

-----
The following sections from the ANS standard, which I thought were
relevant to the problem, are quoted below for your convience:

2.1 Definitions of terms

ambiguous condition:

  A circumstance for which this Standard does not prescribe a specific
  behavior for Forth systems and programs.

  Ambiguous conditions include such things as the absence of a needed
  delimiter while parsing, attempted access to a nonexistent file, or
  attempted use of a nonexistent word. An ambiguous condition also exists
  when a Standard word is passed values that are improper or out of range.

current definition:

  The definition whose compilation has been started but not yet ended.

defining word:

  A Forth word that creates a new definition when executed.

definition:

  A Forth execution procedure compiled into the dictionary.

dictionary:

  An extensible structure that contains definitions and associated
  data space.

3.4.5 Compilation

A program shall not attempt to nest compilation of definitions.

During the compilation of the current definition, a program shall
not execute any defining word, :NONAME, or any definition that
allocates dictionary data space. The compilation of the current
definition may be suspended using [ (left-bracket) and resumed
using ] (right-bracket). While the compilation of the current
definition is suspended, a program shall not execute any defining
word, :NONAME, or any definition that allocates dictionary data space.

6.1.1250  DOES>

does CORE

Interpretation: Interpretation semantics for this word are undefined.

Compilation: ( C: colon-sys1 -- colon-sys2 )

Append the run-time semantics below to the current definition. Whether
or not the current definition is rendered findable in the dictionary by
the compilation of DOES> is implementation defined. Consume colon-sys1
and produce colon-sys2. Append the initiation semantics given below to
the current definition.

Run-time: ( -- ) ( R: nest-sys1 -- )

Replace the execution semantics of the most recent definition, referred
to as name, with the name execution semantics given below. Return control
to the calling definition specified by nest-sys1. An ambiguous condition
exists if name was not defined with CREATE or a user-defined word that
calls CREATE.

Initiation: ( i*x -- i*x a-addr ) ( R: -- nest-sys2 )

Save implementation-dependent information nest-sys2 about the calling
definition. Place name's data field address on the stack. The stack
effects i*x represent arguments to name.

name Execution: ( i*x -- j*x )

Execute the portion of the definition that begins with the initiation
semantics appended by the DOES> which modified name. The stack effects
i*x and j*x represent arguments to and results from name, respectively.

16.3.3 Finding definition names

An ambiguous condition exists if a program changes the compilation word
list during the compilation of a definition or before modification of the
behavior of the most recently compiled definition with ;CODE, DOES>, or
IMMEDIATE.

A.16.3.3 Finding definition names

In other words, the following is not guaranteed to work:

: FOO  ... [ ... SET-CURRENT ] ... RECURSE ... ;  IMMEDIATE

RECURSE, ; (semicolon), and IMMEDIATE may or may not need information
stored in the compilation word list.



Sat, 25 Oct 1997 03:00:00 GMT  
 Defining words, search order and wordlists.
Here's one way to write the code so things are sure to work:


: create-wordlist  ( -- )
   wordlist
   create does-set-wordlist
   dup ,  set-current-wordlist
   1 2 3 fill-wordlist
;

I believe that this avoids all the potential pitfalls that Andrew cited.

Also, there was a bug in Andrew's code:

        create
          wordlist set-current-wordlist
          1 2 3 fill-wordlist
        does>
          set-current-wordlist

That code assumes that new wordlist are created in ALLOTed space, which is
not necessarily true.

The ANS Forth specification of WORDLIST is not entirely air-tight in its
description of where the data structure defining the wordlist lives, but
it is relatively clear that you cannot assume that it is necessarily at
HERE.
        "The new word list may be returned from a pool of preallocated word
        lists or may be dynamically allocated in data space."

That text isn't air-tight because "dynamically allocated in data space" is
not precisely defined.  "dynamically allocated" could be construed to mean
that the data structure must be in some place in data space other than HERE,
i.e. in memory allocated with ALLOCATE, or it could be construed to mean
anyplace at all in data space.  I believe the latter interpretation is
preferable, considering what I know about various common implementation
strategies, and the fact that the committee usually opted in the direction
of leaving implementors a great deal of latitude.

Consequently, it is prudent to make the wordlist first, then create a word
to name it.

Mitch



Sat, 25 Oct 1997 03:00:00 GMT  
 Defining words, search order and wordlists.

Quote:

> Here's one way to write the code so things are sure to work:


> : create-wordlist  ( -- )
>    wordlist
>    create does-set-wordlist
>    dup ,  set-current-wordlist
>    1 2 3 fill-wordlist
> ;

Ok... Given that DOES> is not my strongest suit how/why does this
work?  I've thrown it at PFE and it works and avoids the switching
wordlist problem in my previous mail.  So why does sticking DOES>
in a word by itself get around the problem where the following
DOES> in create-wordlist doesn't:

: create-wordlist  ( -- )
  wordlist dup set-current-wordlist
  1 2 3 fill-wordlist
  previous definitions
  create
    dup , set-current-wordlist
  does>      ( body -- )

Quote:
> Also, there was a bug in Andrew's code:

>    create
>      wordlist set-current-wordlist
>      1 2 3 fill-wordlist
>    does>
>      set-current-wordlist

> That code assumes that new wordlist are created in ALLOTed space, which is
> not necessarily true.

> The ANS Forth specification of WORDLIST is not entirely air-tight in its
> description of where the data structure defining the wordlist lives, but
> it is relatively clear that you cannot assume that it is necessarily at
> HERE.
>    "The new word list may be returned from a pool of preallocated word
>    lists or may be dynamically allocated in data space."

Good point!  I obviously missed that when looking at the standard.


Sat, 25 Oct 1997 03:00:00 GMT  
 Defining words, search order and wordlists.

Quote:
> Ok... Given that DOES> is not my strongest suit how/why does this
> work?  I've thrown it at PFE and it works and avoids the switching
> wordlist problem in my previous mail.  So why does sticking DOES>
> in a word by itself get around the problem where the following
> DOES> in create-wordlist doesn't:

Sticking DOES> in a word by itself allows you to execute DOES> before
create the constants, thus avoiding the issue of the distinction between
the "current" and "most recent" definitions.

One "problem" with the whole design of the DOES> technique, in my opinion,
is that it effectively packages two definitions in one, leading to confusion
and tricky implementation issues.  DOES> effectively "partitions" what appears
to be a single definition into two separate colon definitions.  This is
actually rather bizarre if you think about it.  DOES> simultaneously ends
one word and starts another.  In a lot of cases, DOES> is syntactically
rather convenient, but it isn't a very good factoring of the underlying
capability.

Separating the DOES> clause into a separate word, which can then be called
from the defining word, is a useful workaround for the problem that
DOES> ends the word that contains it.

Mitch



Sat, 25 Oct 1997 03:00:00 GMT  
 Defining words, search order and wordlists.
Andrew Houghton wrote on Tue, 09 May 95 08:09:48 PDT  :

[... text deleted for puposes of this message ...]

Andy,

The behavior you describe of the PFE system is in my opinion
literally conformant with TC intent with regard to interpretation
of "recent" and its selection of "recent" definition in presence
of the wordlist change (an identified ambiguity) is permissible.

The term "recent" literally describes the behavior of most if
not all existing systems.  If a defining word uses EVALUATE or
LOAD to begin new definitions before invoking DOES> then the
last such definition, *temporally*, is the one that DOES>
will apply to.  Hence "recent" as opposed to "current".
Note that *no* definition is implicitly made by a defining
word.  If it makes one, then DOES> alters that, because it
is the most "recent".

In fact, most existing systems given a definition like FOO:

        : FOO   ." I didn't make anything!" DOES> . ;

will, in absence of any error checking, modify the *most recent*
definition created elsewhere to execute as shown after DOES>.

In existing practice with most systems I am aware of this
has nothing necessarily to do with what wordlist is currently
selected, although the exact means whereby those systems keep
track of the *most recent* definition may vary and some might
possibly track this by using wordlist/vocabulary pointers.

I have not checked this aspect of many systems myself.  If there
*are* divergent sensitivities to wordlist selection in making
the determination of recent-ness, the following example (germane
since IMMEDIATE also talks about "recent")

        FORTH DEFINITIONS
        : BOB   ." Bob ran" ;
        : JOE   ." Joe ran" ;  EDITOR IMMEDIATE

would do different things to JOE and BOB.  However, in all
systems I have experience with, the use of the vocabulary
name EDITOR in the example above has no effect on anything
(unless one's EDITOR has a different version of IMMEDIATE).

I believe, but cannot prove without TC clarification, that
the reason why change of wordlist before DOES> or IMMEDIATE
is identified as an ambiguous condition is to allow for
possible implementation differences in the way the "most
recent" definition is kept track of, i.e. per dictionary
instance or per wordlist instance within a dictionary.

Either way, I think that by changing wordlists before
invoking DOES> you have trod upon a defined ambiguous
condition.  However, if wordlists are NOT changed, then
as far as I know a definition

        : MAKE
           " VARIABLE bill" EVALUATE
           " VARIABLE joe"  EVALUATE
           " VARIABLE mike" EVALUATE
              DOES> ." Has been enhanced" ;

will in all conformant systems make three definitions
bill, joe, and mike, all of which act like variables
in stack effect, but mike will in addition display a
message when it's executed.

If you would like a formal TC clarification on this matter
please let me know and we will process your query as such.
This process takes several weeks since it requires not only
composition of a clarification but a letter ballot, but the
TC is cheerfully willing to do this when asked.

    Greg Bailey     |  ATHENA Programming, Inc  |  503-621-3215  |
  ----------------  |  24680 NW Dixie Mtn Road  |  fax 621-3954  |



Sat, 25 Oct 1997 03:00:00 GMT  
 Defining words, search order and wordlists.

Quote:

> : create-wordlist  ( -- )
>   create
>     wordlist set-current-wordlist
>     1 2 3 fill-wordlist
>   does>            ( body -- )
>     set-current-wordlist ;

1) WORDLIST returns a wid (wordlist identifier) which in an ANS Forth
   standard system need not necessarily be equal to HERE.  If the
   word CREATEd above is executed, it passes the address of its data
   area (the former HERE) to set-current-wordlist, but not the wid,
   as you probably desires.

2) Since fill-wordlist will make more recent definitions than the just
   CREATEd word, Does> will not modify the right word but the
   most recent defined word in the new wordlist.

Well, appearantly this led you to:

Quote:
>   \ Create new wordlist with default definitions, revised.
> : create-wl  ( -- )
>   wordlist dup set-current-wordlist
>   1 2 3 fill-wordlist
>   previous definitions
>   create
>     dup , set-current-wordlist
>   does>      ( body -- )


This definition looks ok to me.

Quote:
> The standard seems unclear to me when it says that: "does> replaces the
> execution semantics of the most recent definition."  I would have expected
> it to use the term "current definition" from "2.1 Definitions of terms."
> But again whether its "recent definition" or "current definition" it really
> says to me: "replaces the execution semantics of the most recent/current
> definition [in the most recent/current definition's wordlist]".  

And what is the most recently made definitions in create-wordlist?
The CONSTANT c, built by fill-wordlist. So I think PFE's behaviour is correct.

Quote:
> My final question is: "Is it possible to portably create a defining word
> which creates a wordlist, fills that wordlist with definitions and then
> switches the search order to that wordlist with the current ANS standard?"

What about:

: Empty-Voc ( <name> -- )
    CREATE WORDLIST DUP set-current-wordlist ,

: Voc ( <name> -- )
    Empty-Voc  1 2 3 fill-wordlist ;

Ulrich

--
Ulrich Hoffmann, Uni Kiel        WWW: http://www.informatik.uni-kiel.de/~uho/

Preusserstr 1-9, D-24105 Kiel, Germany      Tel: +49 431 560426   Fax: 566143



Sun, 26 Oct 1997 03:00:00 GMT  
 Defining words, search order and wordlists.

Quote:
> Ok... Given that DOES> is not my strongest suit how/why does this
> work?  I've thrown it at PFE and it works and avoids the switching
> wordlist problem in my previous mail.  So why does sticking DOES>
> in a word by itself get around the problem where the following
> DOES> in create-wordlist doesn't:

> : create-wordlist  ( -- )
>   wordlist dup set-current-wordlist
>   1 2 3 fill-wordlist
>   previous definitions
>   create
>     dup , set-current-wordlist
>   does>      ( body -- )


I threw it to gforth, and it works. But gforth uses a LAST variable to
determine the most recent definition (which is IMHO the intention of the
standard term "most recent"). LAST is set each time HEADER is called to
build a new header. It seems to be that PFE uses the current wordlist to
determine the most recent definition, and with your SET-CURRENT-WORDLIST,
you change it. So the last definition in your new wordlist becomes the
"most recent" definition. That's where it sucks.

Quote:
> > The ANS Forth specification of WORDLIST is not entirely air-tight in its
> > description of where the data structure defining the wordlist lives, but
> > it is relatively clear that you cannot assume that it is necessarily at
> > HERE.
> >       "The new word list may be returned from a pool of preallocated word
> >       lists or may be dynamically allocated in data space."

> Good point!  I obviously missed that when looking at the standard.

I would assume, that the new word list either may be returned form a pool
... or allocated in the dictionary, or allocated elsewhere; but
"elsewhere" is always about the same as "pool".

Bernd Paysan
"Late answers are wrong answers!"
http://www.informatik.tu-muenchen.de/cgi-bin/nph-gateway/hphalle2/~pa...



Mon, 27 Oct 1997 03:00:00 GMT  
 Defining words, search order and wordlists.
Hi Bernd,

Quote:
> ... It seems to be that PFE uses the current wordlist to
> determine the most recent definition, and with your SET-CURRENT-WORDLIST,
> you change it. So the last definition in your new wordlist becomes the
> "most recent" definition. That's where it sucks.

pfe uses a `LAST' variable too. I agree with you that this seems the
clearer concept. The FIG-Forth `LATEST' word is available too but not
used by `CREATE'. My reason for doing it this way was that I don't
like it when after a `FORGET' an old definition suddenly is the ``most
recent'' where words refer to that somehow manipulate this compiled
definition (e.g. `;' flagging a compiled defintion finished.)

Can we somehow detect the error condition here? I mean without
introducing a `<BUILDS' again?

The error condition was

        : DEFINING-WORD
            CREATE
                \ now while seemingly building the body of the
                \ created defintion new definitions are created:
                S" CREATE XYZ" EVALUATE
                \ To make the error less obvious you can change the
                \ wordlist between the two creates.
            DOES>
                \ oops the does-part is attached not to
                \ the defined word from the first but from the second
                \ CREATE inside EVALUATE
                ." I'm a DEFINING-WORD's child " ;

        DEFINING-WORD ABC
        ABC .S
        \ an address of ABC's data field is on stack
        XYZ I'm a DEFINING-WORD's child ok

I don't see how poor DOES> can know which CREATEd word it should refer
to. Resp. why it isn't XYZ but ABC it should refer to. Or even how it
can detect that there is an ambiguity.

Dirk



Mon, 27 Oct 1997 03:00:00 GMT  
 Defining words, search order and wordlists.

Quote:
> Or why couldn't LAST
> be stored in the word list data structure?  After all, definitions are
> associated with word lists.  Why not have a LAST per word list rather
> than a LAST per dictionary?

During one period of time, my system had a per-vocabulary LAST, but I
eventually changed it back to a system-wide LAST.  I don't remember the
exact details of why I had it per-vocabulary, but I think it had something
to do with transient defining words whose DOES> behavior remains resident
in the target system and whose compilation behavior is discarded.

For some reason, something else that I changed made the per-vocabulary LAST
unnecessary, so I changed it back to the system-wide LAST to save space.

It's not a big deal.  It is easy enough to avoid the problem by coding things
in the proper order.

Mitch



Mon, 27 Oct 1997 03:00:00 GMT  
 Defining words, search order and wordlists.

Quote:
Dirk writes:
> Can we somehow detect the error condition here? I mean without
> introducing a `<BUILDS' again?

        A good system will. We have THROWs for things like this that
systems *may* optionally offer. "-51 compilation wordlist changed"
comes to mind. We are allowed, I believe, to define others, as
part of this process.

        Or do you mean "is it possible to detect", well, of course,
you save a lot of state about dictionary operations, you can detect
anything. That's an implementator's option, and I'd recommend it.

                =jax=



Wed, 29 Oct 1997 03:00:00 GMT  
 
 [ 10 post ] 

 Relevant Pages 

1. oops, FORTH-WORDLIST and initial search-order...

2. ANS Standard - Search order words : SET-ORDER Query

3. Inspecting a wordlist's words

4. wordlist-id to wordlist name

5. User defined Ordering in Browser

6. Order of float and float precision contaigon defined?

7. define "variable" word

8. Defining words and POSTPONE

9. Question about newly defined words...

10. Defining words

11. redefining defining words

12. Swapping byte order in blocks of words

 

 
Powered by phpBB® Forum Software