random rant on random files 
Author Message
 random rant on random files

I feel a little upset at Scheme for lack of direct support in
most Schemes for random files.  These are essential for business
programming.  Scheme should and could revolutionize business and
supplant BASIC with all its basic flaws if Scheme provided a
library routine for random files that was part of the definition.

BTW, I think Scheme should go the way of C and make use of
libraries in order to make the core image tiny enough to fit on
Apple IIe's with 64K of memory--that is an ideal worth fighting
for, else Scheme is not a revolution for everyone.  

Any comments that might help me towards my desire to be able to
do real business programming in Scheme???

Cheers!



Mon, 30 Jan 1995 21:24:07 GMT  
 random rant on random files

Quote:

> I feel a little upset at Scheme for lack of direct support in
> most Schemes for random files.  These are essential for ...
> ... core image tiny enough to fit on
> Apple IIe's with 64K of memory-- ...

> Any comments that might help me towards my desire ...

Do you really want to program an Apple IIe? Maybe, just maybe, you could
get SIOD to fit in that with enough space left over for something
interesting.

But seriously, SIOD version 2.8 has fopen,fclose,getc,putc, and puts
as lisp subrs that have semantics like the ones in C.

You should be able to add fread,fwrite,fseek, and ftell without
too much effort.

Then, if you are going to be storing lisp objects in random files,
you probably also want to be using FAST-PRINT and FAST-READ and
not read/print or getc and putc, if what you want to do is
database-like things.

-gjc



Wed, 01 Feb 1995 01:53:28 GMT  
 random rant on random files

Quote:

>I feel a little upset at Scheme for lack of direct support in
>most Schemes for random files.  These are essential for business
>programming.  Scheme should and could revolutionize business and
>supplant BASIC with all its basic flaws if Scheme provided a
>library routine for random files that was part of the definition.
> ...
>Any comments that might help me towards my desire to be able to
>do real business programming in Scheme???

One reason is that random access is not supported is the desire not to
support "files" when persistent data objects are just around the
corner.  My aproach to the random access problem follows.

Enjoy!

;;=====================================================================
; FILE          "store.doc"
; IMPLEMENTS    Proposal for a `Store' data abstraction.
; AUTHOR        Ken{*filter*}ey
; DATE          1991 November 26
; LAST UPDATED  1991 December 12 -- added WRITE-STORE-SLICE

PRIMARY REASON DE ETRA:  Random & Binary i/o.

BASIC IDEA: A (binary) store with size and permissions.

NOTA BENE:  This data type has no print representation.

SYNOPSIS:

(MAKE-STORE num-bytes . fill-integer) -> store
(MAKE-STORE num-bytes fill-integer . permissions) -> store
(MAP-STORE  start end . permissions) -> store
(PERMISSIONS store) -> 'read-only 'read+write or 'write-only
(STORE-SET-PERMISSIONS! store permissions) -> unspecified
(ENDIAN store) -> 'big-endian or 'little-endian
(STORE-LENGTH store) -> num-bytes

(STORE-REF-BYTE store address) -> integer
(STORE-REF-WORD store address) -> integer
(STORE-REF-LONG store address) -> integer
(STORE-REF-CHAR store address) -> character
(STORE-SLICE->STRING store start end) -> string

(STORE-SET-BYTE! store address integer) -> unspecified
(STORE-SET-WORD! store address integer) -> unspecified
(STORE-SET-LONG! store address integer) -> unspecified
(STORE-SET-CHAR! store address character)     -> unspecified
(STRING->STORE-SLICE-SET! string store start) -> unspecified
(STORE-SLICE->STORE-SLICE-SET! store1 start1 end1 store2 start2)
 -> unspecified

(FILE->STORE path) -> store
(STORE->FILE store path) -> unspecified
(WRITE-STORE-SLICE store start end . port) -> unspecified

optional:
  (STORE-REF-IEEE-SINGLE->REAL   store address) -> real
  (STORE-REF-IEEE-DOUBLE->REAL   store address) -> real
  (STORE-REF-IEEE-EXTENDED->REAL store address) -> real
  (STORE-SET-IEEE-SINGLE!   store address real) -> unspecified
  (STORE-SET-IEEE-DOUBLE!   store address real) -> unspecified
  (STORE-SET-IEEE-EXTENDED! store address real) -> unspecified
  ...

DISCUSSION:

The Store abstraction allows one to deal with low-level file,
unscanned-vector, and computer memory `objects', allowing device
drivers and other system programs to be written in Scheme.  An
implementation may supply predefined stores which map to computer
memory or memory mapped devices.  Memory which remains fixed, e.g. C
structs and Pascal records can be mapped as stores and dereferenced.
Memory can be copied to memory, file to file, file to memory, etc.

Permissions are used to allow EPROM memory and read-only files to
signal an error upon assignment.  An error is also signaled if an
integer of greater size than the unit is assigned [byte = 8 bits; word
= 2 bytes; long = 4 bytes].

A Scheme character is of implementation-dependent size (e.g. Japanese
or Chinese characters may be multi-byte quantities).

Returned integers are always non-negative.

ISSUES:

No file extension is provided for.

No support for specifying and mapping between multiple address spaces.

DETAILS:

[Elided -- Hopefully obvious]

;;                      --- E O F ---



Thu, 02 Feb 1995 11:15:25 GMT  
 random rant on random files


Quote:
> I feel a little upset at Scheme for lack of direct support in
> most Schemes for random files.

Without in any way endorsing the original poster's desire to make
Scheme into the new COBOL :):), I too was surprised and disappointed
that R4RS Scheme doesn't have a seek function to apply to ports.  I
have an application where I need to read a (large) file line-wise
backwards, and the only Scheme I can code it in at the moment is
Scheme->C where I have access to libc.  That's great, but hardly
portable.  Does anyone know whether this is being considered for
the future of Scheme?  On a more general note, is there any way
(perhaps a mailing list to subscribe to, or something) that people
in general can find out what _is_ being thought about for R5RS?

--

`No Franz Lisp executable file should EVER be stripped.'
        - Franz Lisp manual, VAX UNIX



Fri, 03 Feb 1995 18:07:35 GMT  
 random rant on random files
I don't find this "store" idea obvious at all.  Does it live in memory
or on the disk?  If on the disk, does that mean there is a redundant
disk-to-disk copy when I finally use store->file?  If in memory, how
do you handle gigabyte stores?

I'm not necessarily objecting to the idea.  I just need to see more
discussion than was posted about why this is thought preferable to
just adding seek, binary-read, and binary-write to the port mechanism.



Fri, 03 Feb 1995 22:26:46 GMT  
 random rant on random files

Quote:

>I don't find this "store" idea obvious at all.  Does it live in memory
>or on the disk?  If on the disk, does that mean there is a redundant
>disk-to-disk copy when I finally use store->file?  If in memory, how
>do you handle gigabyte stores?

I think that the point of it is to _deliberate_ fuzz the issue of
where the store is located. It doesn't matter! All that matters is the
behavior of a store when you use it - the rest is implementation
details.

Given a system like a store, what's great about it is that it can
provide read/write access to _anything_: a store can be in memory, on
a disk, a combination... it can be anything at all that behaves the
correct way, and the implementation of stores is responsible for
providing an appropriate low-level implementation.

(For an example of how well ideas like this can work, take a look at
the Modula-3 I/O streams system. It provides perfectly flexible,
general access to _anything_ that can be read or written.)

Quote:
>I'm not necessarily objecting to the idea.  I just need to see more
>discussion than was posted about why this is thought preferable to
>just adding seek, binary-read, and binary-write to the port mechanism.

It's preferable because it doesn't rely on the filesystem, and it
isn't specific to disks. A store can be used to read/write a section
of memory, or a file on disk, or a screen display, or...

        <MC>

--
|| Mark Craig Carroll: <MC>     || "There is no such thing as a problem
|| Univ of Delaware, Dept of CIS||  without a gift for you in its hands. You
|| Grad Student/Labstaff Hacker ||  seek problems because you need their



Sat, 04 Feb 1995 02:09:04 GMT  
 random rant on random files

Quote:

>I don't find this "store" idea obvious at all.  Does it live in memory
>or on the disk?  If on the disk, does that mean there is a redundant
>disk-to-disk copy when I finally use store->file?  If in memory, how
>do you handle gigabyte stores?

Sorry.  Let me amplify a bit.

The creation operations return a "store" object.  All store objects
are treated the same.  For files, the bits stay on the disk (mod
buffering).  STORE->FILE writes the contents of a store (the bits)
into a file.  If the store represents a file, then you are doing a
file->file copy.  If the store represents a slice of memory, you are
doing a memory->file copy.  You have full random, binary access to
store objects, so (STORE-REF-BYTE aStore 37) reads a byte from a file
or a memory location at "start"+ 37.

As stores are random, binary i/o, everything is mapped to positive
integers.  If you want to write a Scheme object (e.g. a vector or a
negative integer, etc.) it is up to you to do the representation
mapping.

To map memory to a store: (MAP-STORE 100 200) returns a store with
a base address of 100.  (store-ref-byte thisStore 0) gives you the
byte at address 100+0 in your memory space.  Thus you can build a
structure mechanism on top of stores to get values of C structs,
Pascal records, whatever, by name.

You can also use stores which are really vectors unscanned by the
collector.  This is what MAKE-STORE returns.

If you have not guessed by now, stores are a pretty low-level
interface.  If done portable, however, this just happens to be a
higher-level interface than we have now.  I think that the abstraction
is useful.

So let me try again.

Quote:
> Does it live in memory or on the disk?

It does not matter to the user of the abstraction, but memory is
probably in memory and files are probably on disk (mod buffering).

Quote:
> If on the disk, does that mean there is a redundant
>disk-to-disk copy when I finally use store->file?

If you are doing a copy, where is the redundancy?

Quote:
> If in memory, how do you handle gigabyte stores?

By doing *nothing* special {I assume bignums and linear addressing}.
I am not copying anything to create a store (just as opening a file
does not copy the file).

Please ask more questions where confused.
-Ken



Sat, 04 Feb 1995 07:00:56 GMT  
 random rant on random files

   The creation operations return a "store" object.  All store objects
   are treated the same.  For files, the bits stay on the disk (mod
   buffering).  STORE->FILE writes the contents of a store (the bits)
   into a file. [...]  As stores are random, binary i/o, everything is
   mapped to positive integers.  If you want to write a Scheme object
   (e.g. a vector or a negative integer, etc.) it is up to you to do
   the representation mapping.

It seems to me that stores are the usual binary streams, but that you
are required to give an offset (a "seek" address) with every
read/write operation.

Personally, I prefer the old-fashioned binary stream interface; it is
the natural interface to things that may or may not be random access
and that may or may not be bidirectional. Stores look too much like
binary arrays, and, to me, the interface does not clearly reflect the
costs of the underlying operations.

What is wrong with binary streams?

                                Thomas.



Sun, 05 Feb 1995 05:12:57 GMT  
 random rant on random files

Okay, so what do I do if I want to write a new file from scratch?
I don't want to create a store, buffer it in a temp file on disk,
and then finally copy it to the filename I really meant.  (That's
what I was trying to ask about copying.)

The part about unsigned-integers-only seems a little user-unfriendly
to me.  To make this work in a way that would be both convenient and
Schemely, it seems to me (thinking about it off the top of my head)
that we'd need -- dare I say it -- first-class types!  Then you could
say (datum->store <datum> <type> <store-addr>) and
(store->datum <type> <store-addr>).  You'd want the first-class type
mechanism to include user-defined ADTs as well as the primitive types.

I mean, how does someone write a bignum into a store?  This seems
way too hairy for a simple-minded programmer like me; I'd rather
do the I/O in C!  (Which I hate, if that wasn't obvious.)

Alternatively, maybe every primitive type has to have an unambiguous
binary representation the same as it has a print form, and instead of
explicitly mentioning types you just leave that argument out and
Scheme figures it out.  That would mean, though, tagging everything
in the store with its type.

I'm feeling a little disorganized about this message, but it's because
I'm really struggling to make sense of the whole idea.  I mean, files
just feel very different from memory to me.  I know that at some level
of abstraction they're the same, but they perform quite differently.
I don't understand [I'm not being sarcastic] the principle of language
design whereby symbols and strings are treated separately but files
and memory are treated the same.



Sat, 04 Feb 1995 23:41:20 GMT  
 random rant on random files
The STORE concept is a Scheme adaptation of memory-mapped files.
Nothing more.

--

Lance Norskog

Data is not information is not knowledge is not wisdom.



Sun, 05 Feb 1995 00:47:00 GMT  
 
 [ 29 post ]  Go to page: [1] [2]

 Relevant Pages 

1. Opening random line from file with random.shuffle()

2. A truly random $random??

3. ? generating random uniform and binomial random deviates for BIG integers

4. Unit tests and random.Random()

5. Random number not random?

6. random behavior in random module?

7. Random Number Generator to produce SAME random number from 12:00am-11:59pm

8. How to compile Ada.Text_IO.xxxxx files (Was: Random Access in Ada.Text_IO files)

9. Random Access file reading in J...

10. VW7: random access file IO?

11. Random Line from a File

12. Random printing from a file , help please

 

 
Powered by phpBB® Forum Software