Reading files 
Author Message
 Reading files

Is it not possible to read a file at once instead of using read-line
or read-char? The following is very slow for big files.
I'm working with GCL under Linux. Thanks.

(defmacro readfile (name)
`(with-open-file (file ,name :direction :input)
(do ((row (read-line file) (read-line file nil 'file-end))
(text "" (format nil "~a~a~%" text row)))
((eql row  'file-end) text))))

J. Eschgfaeller



Thu, 15 Nov 2001 03:00:00 GMT  
 Reading files

| Is it not possible to read a file at once instead of using read-line
| or read-char?

  two ways: (1) use READ-SEQUENCE on a pre-allocated string (use
  FILE-LENGTH to learn how big it should be), or (2) make a FFI call to
  the mmap system call and treat it as a string.  the first is simple.

| I'm working with GCL under Linux. Thanks.

  there are much better Common Lisp implementations available under Linux.
  I would recommend Allegro CL from Franz Inc.  visit www.franz.com.

#:Erik



Thu, 15 Nov 2001 03:00:00 GMT  
 Reading files

Quote:

> Is it not possible to read a file at once instead of using read-line
> or read-char? The following is very slow for big files.

The code you uses uses an O(n^2) string copying, so it's not surprising
that it's slow.  

Quote:
> I'm working with GCL under Linux. Thanks.

> (defmacro readfile (name)
> `(with-open-file (file ,name :direction :input)
> (do ((row (read-line file) (read-line file nil 'file-end))
> (text "" (format nil "~a~a~%" text row)))
> ((eql row  'file-end) text))))

> J. Eschgfaeller

(There's also zero justification for this being a macro.  It will only
bloat your code and the savings in speed is negligible.  A function call
is probably set up in only a couple of instructions, and anything
that is doing this many calls to read-line, open, format, etc. is using
so many other instructions that the value of having this be a macro
rapidly approaches zero.)

Even without doing anything else, you could switch it to
an O(n) algorithm (where n is the number of lines) by doing:

 (defun read-file (filename)
   (with-open-file (stream filename :direction :input)
     (read-stream stream)))

 (defun read-stream (stream)
   (with-output-to-string (result-stream)
     (loop (multiple-value-bind (line eof-p)
               (read-line stream nil nil)
             (when line (write-string line result-stream))
             (when eof-p (return))
             (terpri result-stream)))))

However, if you need greater efficiency, you should consider using
READ-SEQUENCE instead of READ-LINE, so that you can buffer larger
blocks of text than single lines.

See CLHS for info on READ-SEQUENCE.
http://www.harlequin.com/education/books/HyperSpec/FrontMatter/index....



Thu, 15 Nov 2001 03:00:00 GMT  
 
 [ 3 post ] 

 Relevant Pages 

1. Q: simple example of read file->convert char(s)->write file needed

2. file io with verilog, reading file into a reg

3. Q: Reading file in multiple times not working!

4. Question about reading files

5. read file into string

6. URGENT reading file sequentially - driving me to despair

7. Cannot read file over network

8. help sql error 47 in read file in program

9. Cant read file in dictionary

10. Help Reading File

11. Can I read files made in version 4.5 with 4.1 Reader

12. Reading files backward

 

 
Powered by phpBB® Forum Software