Forwarding message 22.03.00 from Jonathan Cunningham 
Author Message
 Forwarding message 22.03.00 from Jonathan Cunningham

Another message that did not get through the Mail->News link.
Aaron



Subject: Re: forwarding message from Jonathan Cunningham (with comments)
Date: Wed, 22 Mar 2000 23:17:19 -0000

Steve Leach (Hi, Steve) says about environment variables:

Quote:
>Actually, I understand that there is a good reason.  Apparently it
>helps the Poplog devteam work with multiple versions.  However,

I was resisting the urge to followup on this, but since I'm replying
to Aaron (below), I'll make the same point here. I would prefer to
say, not that "there is a good reason" but that "there was a good
reason".

Actually, with 20-20 hindsight (and inspired by the recent discussion
on this group), I think it is more accurate to say that the zillion
environment variables are one solution to a problem, and that poplog
is much better off with some solution than no solution.

But my point, and I think I'm just trying to make clear what people
(including Steve) were already saying, was that there are alternative
solutions, which might be more appropriate.

To summarise:
  Houston, we have a problem.
  Squintillions of environment variables can solve the problem.

Does it follow that therefore we need squintillions of environment
variables?

The correct answer is "no". If you answered "yes", can you see
your error?

Yes, of course you can. All you need to do is ask yourself, "How can
we solve this problem, in a poplog way, without using environment
variables?"

Internally, pop11 may need to distinguish skillions and whillions of
directories and search paths. The problem is simply to initialise some
internal data structure (or set of variables) with this information.

There is no reason why, when poplog starts up, it cannot read this from
a configuration file. The programming language "pop11" is quite
powerful enough for this purpose, and so the configuration file could
be written in pop11 (although I would probably design something
language-neutral).

This has already been stated by other people earlier in this discussion.

Of course, reading two dozen lines of pop11 code will slow down
startup by several milliseconds. Added up over future centuries of
poplog users, this could add up to minutes of human impatience. On
the other hand, you probably gain back more than it costs.

I don't think I'm saying anything that wasn't implicit in what both
Andrew and Steve have already said. I'm just trying to step back and
make it clearer.

Now I'll reply to Aaron's comments:

Quote:
>1. Poplog, unlike linux default mechanisms, was always designed so that
>one can have more than one version installed simultaneously. So a new

Indeed. And environment variables were one way to achieve this.

Quote:
>version could be installed on a system for testing by experts and brave
>people while the old version remains available for the rest. Later the

And a configuration file would let you do the same thing, without
knowing what an environment variable was.

Quote:
>2. I don't know all the reasons why poplog has so many environment
>variables, but one reason they can be useful is if a common file server

Perhaps because it seemed like a good idea when there were only two
or three? And the number just grew and grew??

(snip)

Quote:
>An alternative to having lots of environment variables in order to allow
>parts of the directory tree to be changed for various purposes, while

This is another mechanism which could, perhaps, be made to work.

I prefer the approach of a (pop11?) configuration file, with two dozen
assignments to (commented) variables, or (just perhaps), something
like an association list or mini-pop11 "database," you know,
something like:
  [[auto ['/var/bin/poplog/lib/auto' '/users/aaron/auto']]
   [xvedstuff ['/var/bin/poplog/lib/ved/xved']]
   ...
  ] -> variable_which_replaces_lots_of_environment_variables;

This is much too simplistic, so don't feel you need to point that out.

I'm merely reiterating the point(s) I made above: you don't need
environment variables. They are only a good solution when they are
a good solution.

The rest of Aaron's comments seem to me to be explaining, justifying
and defending the current mechanism, with suggestions for how it can be
improved.

In the interest of clarity, let me be explicit: it seems to me that
the drift of the discussion was not that the environment variable
mechanism needs to be improved, but that it could be replaced by a
purely poplog mechanism, and that this might have advantages.

To run poplog, even when there are multiple overlapping installations,
and many different users, you need two pieces of information:
(1) Which executable binary file to run (newpop, pop11 or whatever)
(2) Where your individual configuration information can be found.

If you were designing a solution to this from scratch, it is unlikely
you would invent the current mechanism. So if you wish to defend the
current mechanism, you have to be able to justify why a more elegant
solution cannot be used. "The current mechanism works" is not
sufficient justification to rule out discussion of alternatives.

Jonathan



Tue, 10 Sep 2002 03:00:00 GMT  
 Forwarding message 22.03.00 from Jonathan Cunningham

Steve Leach and by me.

I don't disagree with anything Jonathan wrote, though probably because I
did not explain clearly what I was getting at, he mistook my intent in
my paragraph on multiple versions.

Quote:

> Date: Wed, 22 Mar 2000 23:17:19 -0000

> Steve Leach (Hi, Steve) says about environment variables:

> >Actually, I understand that there is a good reason.  Apparently it
> >helps the Poplog devteam work with multiple versions.  However,

> I was resisting the urge to followup on this, but since I'm replying
> to Aaron (below), I'll make the same point here. I would prefer to
> say, not that "there is a good reason" but that "there was a good
> reason".

I'll go along with that, and in my message pointed out that for most of
those "good" reasons the sort of alternative suggested previously by
Andrew Sayer would work (i.e. setting up popenvlist, or various other
lists, after poplog starts up, in the basis of some sort of central
or user initialisation file).

However, different reasons need different treatment.

Quote:
> Actually, with 20-20 hindsight (and inspired by the recent discussion
> on this group), I think it is more accurate to say that the zillion
> environment variables are one solution to a problem,

To be more precise, Poplog uses different environment variables as
solutions to different problems, some more important than others.

Quote:
> ...
> Internally, pop11 may need to distinguish skillions and whillions of
> directories and search paths. The problem is simply to initialise some
> internal data structure (or set of variables) with this information.

> There is no reason why, when poplog starts up, it cannot read this from
> a configuration file.

I think we have all now accepted that point regarding most of the
environment variables.

However, some of the environment variables are used to determine which
version of poplog to run. One is $usepop, and the other is the standard
Unix environment variable $PATH. I used to think both needed to be set
up before one gave any command to run poplog, but I now suspect not.
I'll comment on $PATH below, and on $usepop in another message.

There's another group of variables used to determine not which version
of poplog, but which language system or saved image you want to run,
e.g. basepop11, pop11, prolog, clisp, pml, etc.

I'll elaborate on some options for that in a later message.

Quote:
> The programming language "pop11" is quite
> powerful enough for this purpose, and so the configuration file could
> be written in pop11 (although I would probably design something
> language-neutral).

Yes. We don't need the extra syntax that pop11 would require. E.g.
lines of name value pairs separated by space or '=' would suffice.

(Compare the difference between vedsetkey, which is a procedure and
requires pop11 syntax, and vedset, which is a syntax form defining a
language-neutral way of specifying ved's key bindings, screen behaviour,
etc.)

We need something similar for pop-11's and Xved's defaults, instead of
the horrible (in my view) requirement to set things in an external
general purpose X defaults file, the point of which I've never
understood.

Quote:
> I don't think I'm saying anything that wasn't implicit in what both
> Andrew and Steve have already said.

And I had agreed also!

Quote:
> ...
> Now I'll reply to Aaron's comments:

> >1. Poplog, unlike linux default mechanisms, was always designed so that
> >one can have more than one version installed simultaneously. So a new

> Indeed. And environment variables were one way to achieve this.

When I made that point it was not intended to be a comment on the
environment variables that are unique to poplog, but a comment on the
notion of 'the linux way' of doing things, a notion which I think Andrew
had brought up.

And as I understand it the linux way of doing things (unlike Solaris,
for instance???) is that for a complex new package all executables go in
the same bin directory (e.g. /usr/local/bin/, all stuff to do with
libraries go in the same lib directory e.g. /usr/local/lib/ or maybe
/usr/X11/lib, all man files in /usr/local/man, etc.

So that means that when you install a package you scatter files all over
the place, and that makes it hard to have simultaneously a new and an
old version of the package given that all the files have the same names
in both.

I was contrasting that way of installing a package with keeping
everything relating to the package in ONE directory tree and altering
the standard "path" variables, e.g. $PATH, $MANPATH etc.

That not only allows different versions to coexist but also simplifies
addition and deletion. You don't need to keep a table of things to
remove when you remove a package: you simply delete the directory tree.
If a new version becomes the current one, then a single file "sourced"
by all users on the system, can be edited to specify the root of the new
tree.

So the poplog way of doing things, keeping everything in one tree,
requires a few existing Unix environment variables to be changed.

I believe that that's far better for most users. Steve, whose concerns
are different, e.g. use of pop11 in system startup scripts, disagrees,
and thinks that in order to be used as a system language it will have to
do what the others do, e.g. perl, gcc, or whatever.

HOW TO DO IT
Assuming you have different versions of poplog (e.g. an old one, and
a new one that's on trial for a while, for use by experts) in different
directory trees there is the problem of how users gain access to the one
they want. Let's assume for now that the problem of setting appropriate
values for $usepop $popexternlib, etc. etc. can be solved once poplog
starts up: how do you start up the one you want?

Method 1
Obviously one way is to extend $PATH to include the directory containing
poplog executables, which is what happens now, in
    $usepop/pop/com/poplog
    $usepop/pop/com/poplog.sh

(Incidentally the directory is called 'com' not 'bin' because poplog
started life on a Vax running VMS, having previously been a very much
simpler pop11 executable on a tiny PDP-11/40 computer running Unix.)

Method 2
If you don't want to have to change $PATH, to specify the version of
poplog you want, another way is to put symbolic links with different
names in the bin directory, for instance. Thus /usr/local/bin/pop11
could be a symbolic link to the old version of pop/pop/basepop11, and
/usr/local/bin/pop11.new (or something else) could be a symbolic link to
the new version).

And likewise clisp, clisp.new, pml, pml.new etc. This could work, and is
only a little messy. If the "man" files change with different versions,
it starts getting too messy.

That has the advantage over the use of environment variables and $PATH
that you can quickly and easily switch between running pop11.new and
pop11, whereas now you have the {*filter*} task of unsetting environment
variables, then re-setting them (for which I have a shell script copied
from sussex, if anyone wants it!).

I.e. in this approach, instead of scattering around a lot of files, you
scatter symbolic links (and make sure that initialisation files within
the different versions do the rest).

Then deleting an old version means simply deleting ONE directory tree,
though you may have some dangling symbolic links if you don't put
something else in its place.

Installing a new version which does not replace an old version, then
requires installing or changing some symbolic links to point to the new
version.

However, if you have to keep track of too many symbolic links to
change/remove, it can be almost as bad as scattering the files. (I've
not tried to work out how many would be needed: probably fewer than 10).

Method 3
There is another obvious way to do it, which I had not previously
considered, which avoids having links with different names for old and
new versions. That is to have a system-wide standard directory for new
versions of packages, e.g. /usr/local/newbin/ which is by default on the
$PATH of everyone interested in *new* versions. Then the pop11,
basepop11, clisp etc. in that directory are all symbolic links to the
corresponding files in the new version of poplog, and the new versions
will simply shadow the links in /usr/local/bin to the "current" version.

But that solution requires users to buy into all the "new" versions of
all packages, which some will relish and others will not.

Another variant of that would be to have different classes of "newbin"
directories, and then one could select which ones to put on one's $PATH.
I don't know how many Unix "path" environment variables would need to be
handled this way (MANPATH, LD_LIBRARY_PATH, ???). But doing that may be
not much better than just putting onto $PATH additional directories
of executables.

Method 4.
Yet another solution is a variant of the poplog shell script previously
announced, which can go into a standard bin directory and which sets
environment variables or uses some other mechanism to select the version
wanted.

Then a script called newpoplog could be run as
    newpoplog pop11
    newpoplog clisp
    newpoplog xved

etc.

This, like the symbolic links with different names in Method 2, makes it
very easy to switch between new and old versions.

But several people have already objected to starting up Poplog via
shell scripts.

There are probably other solutions people will think up.

And there may be people who are sure they will never want to have two or
more versions installed on their machines and simply don't want to have
to put up with mechanisms whick support that option.

(Just wait till Poplog V16 is released, with all the changes now being
discussed!!).

[AS]

Quote:
> >version could be installed on a system for testing by experts and brave
> >people while the old version remains available for the rest. Later the

[JCL]
> And a configuration file would let you do the same thing, without
> knowing what an environment variable was.

...

read more »



Wed, 11 Sep 2002 03:00:00 GMT  
 Forwarding message 22.03.00 from Jonathan Cunningham
[To reply replace "Aaron.Sloman.XX" with "A.Sloman"]

One of the points made earlier is that not all Poplog's environment
variables are for search lists. Some are there as part of a solution
to the problem of starting up in different modes.

This message explains what happens, and at the end reports on an
experiment to replace environment variables set for each user with an
initialisation file.

Then just one environment variable suffices: but when used in that mode
pop11/ved/xved etc. can't be given any arguments when started up, unless
the system is changed. And maybe it is simpler to use a shell script
that sets the environment variables then

THE CURRENT MECHANISM FOR CHOOSING SYSTEM

When poplog is run, the default at present is that you always have to
run the basepop11 executable, and then on top of that you run one or
more saved images, depending whether you want pop11, prolog, lisp, pml,
xved, etc.

By default the saved images go in $popsavelib, i.e.
    $usepop/pop/lib/psv/

though they can also go in $poplocalbin, which at Birmingham is
defined to be this directory
    $usepop/poplocalbin

For pop-11 the saved image to be run is called startup.psv. I.e.

    basepop11 +$popsavelib/startup.psv

is (normally) invoked simply as pop11.

The other systems available as saved images are usually
    Common Lisp, invoked as clisp, i.e.
        basepop11 +$popsavelib/startup.psv +$popsavelib/clisp.psv

    Prolog invoked as prolog,
        basepop11 +$popsavelib/startup.psv +$popsavelib/prolog.psv

    Standard ML invoked as pml
        basepop11 +$popsavelib/startup.psv +$popsavelib/pml.psv

    XVed, invoked as xved
        basepop11 +$popsavelib/startup.psv +$popsavelib/xved.psv

These saved images are all created (or re-created) by running
files in the $popcom directory
    $popcom/mkstartup
    $popcom/mkclisp
    $popcom/mkplog
    $popcom/mkpml
    $popcom/mkxved
and more.

Nobody would wish to start up an editor by having to type:

        basepop11 +$popsavelib/startup.psv +$popsavelib/xved.psv

One of the uses of environment variables in Poplog is to make it easy
to run a particular system, with basepop11 and one or more saved images
on top of it, in a single command.

This had nothing to do with the requirements of the development team,
but was (rightly or wrongly) thought to be convenient for users.

One possible solution would have been to provide a set of aliases, e.g.
    pop11 =  basepop11 +startup.psv
    prolog = pop11 +prolog.psv
    clisp =  pop11 +clisp.psv
    etc.

However, in the earliest unix shell, 'sh' one could not define aliases
or functions.

John Gibson hit on the following solution, which works no matter which
shell you use.

Allow the basic executable to be run under various names (ie. files with
different names linked to basepop11, e.g. pop11, prolog, clisp, xved,
etc.).

Then when the process starts up it looks to see how it was invoked. If
the process was invoked using the file called XXX, it then looks
to see if there is an environment variable pop_XXX, and if that
exists it acts as if the program had been invoked as
    basepop11 $pop_XXX

So if you have a saved image called XXX.psv and you want to run it using
a command XXX, then

1.  Use ln -s to make XXX a symbolic link in a directory in your
    $PATH to $popsys/basepop11
2.  define pop_XXX as an environment variable equivalent
to
        +startup.psv +XXX.psv

3. type: rehash

Then if you give the command XXX, that will be equivalent
to giving the command

    basepop11 +startup.psv +XXX.psv

Moreover, you can add extra arguments.

This solution works for ALL The unix shells, since they all allow
Unix environment variables to be defined (though with different syntax).

There is another type of environment variable provided for easy startup.

If you wish to start up pop11 and immediately have Ved start editing a
file myfile.p, you can type:

    pop11 ":sysinitcomp();ved myfile.p"

So, to avoid having to type all that you can define the file "ved"
as a symbolic link to $popsys/basepop11, and then define the environment
variable pop_ved as equivalent to

     "$pop_pop11 :sysinitcomp();ved"

Then typing "ved myfile.p" will start up pop11, make it go immediately
into ved with myfile.p as argument. (If your vedinit.p file has
    "x" -> vedusewindows;
then it will always start up in Xved.)

All this explains some of the environment variables defined in these
files

    $popsys/popenv
and
    $popsys/popenv.sh

All that is not a *defence* of the current system, but a *description*.
It is explained more concisely in REF SYSTEM.

Following earlier discussions it is now clear that instead of using
environment variables basepop11 could be made to read an initialisation
file, that specifies mappings from the name by which it was invoked
(poparg0) onto a collection of layered saved images to be restored
or other special startup actions.

So when I previously suggested that this particular use of
environment variables could not be replaced by a file that is read when
basepop11 starts up, I was wrong. It could be done. Doing it properly
would require some re-design of the core of poplog.

I have been experimenting with a scheme that gets rid of all environment
variables except one, $popsys.

This uses the fact that when basepop11 starts up, IF THERE ARE NO
COMMAND LINE ARGUMENTS it does this:

    trycompile( '$popsys/init.p' ) ->;

    unless trycompile( '$poplib/init.p' ) then
        trycompile( 'init.p' ) ->;
    endunless;

This means we could put something in $popsys/init.p to set environment
variables (as previously suggested by Andrew Sayer) and ALSO to use
poparg0 to decide which saved image to run. At first I tried using
sysrestore, but that didn't work. It produces

    ;;; MISHAP - sys_timer: NO SPACE LEFT FOR TIMERS
    ;;; DOING    :

I have no idea why.

But then I realised that sysexecute could be used, to replace the
current process with a new one, with a slight efficiency loss.

So I now have an experimental pair of files
    $popsys/init.p
    $popsys/popenv.p

The first one (init.p) simply contains this:

=======================================================================
#_IF systranslate('ARGSDONE')

false -> systranslate('ARGSDONE');

#_ELSE

trycompile('$popsys/popenv.p') ->;

#_ENDIF
=======================================================================

The second file popenv.p (based mainly on $popcom/popenv and
$popsys/popenv) works out what $usepop $popexternlib and all the other
required environment variables should be (on the basis of $popsys) and
also if necessary sets up $LD_LIBRARY_PATH (required on Suns) or
$SHLIB_PATH (required on HP-UX). It sets up all those environment
variables, and adds ARGSDONE=1 as an additional environment variable.

It then uses a list mapping invocation names, e.g. 'pop11', 'prolog',
'xved', etc. to a collection of arguments to give to basepop11.

Then it uses sysexecute to run basepop11 with %nobanner and the
extra arguments and with all the environment variables set up.

When that happens the init.p file is compiled again, but this time
$ARGSDONE is set, so it simply unsets it and continues.

On a 140 Mhz ultrasparc 1, this mechanism takes no more than about
0.2 seconds longer to start up than the normal mechanism using
environment variables. On a Dell 400 mhz celeron Laptop running RedHat
linux, the difference in startup time is much smaller. It is certainly
not noticeable in either case.

Both the init.p and the popenv.p file can be looked at in here:

    http://www.cs.bham.ac.uk/research/poplog/sysinit/

They are both in this tarball
    http://www.cs.bham.ac.uk/research/poplog/sysinit.tar.gz

There is one serious problem. This mechanism is incompatible with giving
pop11, or ved, or xved, etc. any arguments when it starts up. That's
because if basepop11 is run with arguments it will not try to compile
$popsys/init.p

Hence all my code is by-passed. I have not found any way to pass
arguments in: when there are arguments the init.p file is not read.

So if you are prepared to run poplog/pop11/xved with no arguments
you can use ONE environment variable, $popsys, and put the init.p and
popenv.p into the directory of that name.

However, it seems to be more flexible to revert to an earlier suggestion
and run poplog via shell script which does

    setenv usepop ...
    source $usepop/pop/com/poplog

    exec $*

The use of "exec" means that the shell process does not hang around: the
poplog process takes its place, and inherits all of the environment
variables.

In my tests this is pretty quick. I think it works equally well no
matter which shell you are using to start with. And it allows arguments
to be passed to pop11, xved, etc.

Moreover, it too can be invoked from a script by using exec. E.g.
=======================================================================
#!/bin/sh

exec poplog pop11 << \\\\

'Hello world' =>

\\
=======================================================================



Sat, 14 Sep 2002 03:00:00 GMT  
 Forwarding message 22.03.00 from Jonathan Cunningham

Quote:
> However, it seems to be more flexible to revert to an earlier suggestion
> and run poplog via shell script which does

>     setenv usepop ...
>     source $usepop/pop/com/poplog

>     exec $*

One small change would be to do something like:

exec $popsys/`echo $0 | awk -F/ '{print $NF }'` $*

Which would allow you to do '/usr/bin/foo' instead of '/usr/bin/poplog
$popsys/foo'

I'll start looking into some kind of compromise solution that uses more
(but still few) variables, but no scripts, but other than that this seems
like a good solution.

Incidentally, how does winpop11 get by with just a binary?  Is that
solution something that can be moved across?

        - Andrew



Sun, 15 Sep 2002 03:00:00 GMT  
 
 [ 6 post ] 

 Relevant Pages 

1. Followup to message 22.03.00 from Jonathan Cunningham (fwd)

2. forwarding message from Jonathan Cunningham (with comments)

3. HEX$ { 00 01 -- 02 03 -- 04 }

4. cursor behaviour rmcobol 7.00.03

5. Fw: FWD: from Jonathan Cunningham about Macintosh implementations

6. news-relayFrom: jlc@uk.co.bmtech (jlc (Jonathan Cunningham))

7. ANNOUNCE: GRIDPLUS 1.00.00 - A Grid Based Screen Building Tool For TCL/TK

8. How to retrive forwarded message from multipart message body

9. F83 for SUNs (Part 22 of 22)

10. Ada News - Week Ending: December 22, 1995 - 95-12-22.txt [1/1]

11. Reuse News - Week Ending 22 September 1995 - 95-09-22.txt [1/1]

12. Forwarded message from patterns mailing list...

 

 
Powered by phpBB® Forum Software