How to do fast sequential binary read ? 
Author Message
 How to do fast sequential binary read ?

I've a code that works:

procedure SomeFct (s: filename);
var
  ch: char;
  cnt: integer;
  f: File;
begin
  AssignFile (f, filename);
  { $I- } Reset (f, 1); { $I+ }
  if IOResult = 0 then begin
    try
      { $I- } BlockRead (f, ch, 1, cnt); { $I+ }
      while (IOResult = 0) and (cnt = 1) do begin
         ... process ch ...
         { $I- } BlockRead (f, ch, 1, cnt); { $I+ }
      end;
    finally
      CloseFile (f);
    end;
  end;
end;

but it is horribly slow, because using BlockRead() seems
to bypass all buffering mechanisms, calling the systems
read() functions all the time. How may I speed this up ?

Using 'Read (f, ch)' doesn't work, I only get a cryptical
error message that this is not allowed.

Thank you very much in advance !



Tue, 21 Oct 2003 14:48:01 GMT  
 How to do fast sequential binary read ?

Quote:
> but it is horribly slow, because using BlockRead() seems
> to bypass all buffering mechanisms, calling the systems
> read() functions all the time. How may I speed this up ?

> Using 'Read (f, ch)' doesn't work, I only get a cryptical
> error message that this is not allowed.

You gave the answer yourself: Buffer.

Pseudo code:

var buffer:array[0..4095] of char;      {or byte. Larger buffers hardly
speed up}
.
.
  repeat
   blockread(f,buffer,bytesIWantToRead,BytesActuallyRead);
   [process "BytesActuallyRead" bytes of data in buffer}
  until BytesActuallyRead<>BytesIWantToRead;

BytesIWantToRead is the size of the buffer 4096 in this case.



Tue, 21 Oct 2003 16:46:25 GMT  
 How to do fast sequential binary read ?

Quote:
>I've a code that works:

Type inbuftype = array[1..32768] of char;

var inbuf : inbuftype;

Quote:
>procedure SomeFct (s: filename);
>var
>  ch: char;
>  cnt: integer;
cnt : word;
>  f: File;
>begin
>  AssignFile (f, filename);
>  { $I- } Reset (f, 1); { $I+ }
>  if IOResult = 0 then begin
>    try
>      { $I- } BlockRead (f, ch, 1, cnt); { $I+ }

BlockRead(f, inbuf[1], sizeof(inbuf), cnt);
Quote:
>      while (IOResult = 0) and (cnt = 1) do begin

While (ioresult = 0) and ( cnt <> 0 ) do begin
Quote:
>         ... process ch ...

Use a for loop using cnt to read each char from the buffer
Quote:
>         { $I- } BlockRead (f, ch, 1, cnt); { $I+ }
>      end;

>but it is horribly slow, because using BlockRead() seems
>to bypass all buffering mechanisms, calling the systems
>read() functions all the time. How may I speed this up ?

You can use BlockRead to read in more than one character at a time by
specifying a buffer, using the start of the buffer as above, and the
sizeof function for the number of characters to read. As you have
specified a record size of 1 in your rest the above will read 32768
bytes into memory at a time. You'll find it far quicker to process
an array of chars in memory than reading individually off the disk.

HTH

--
Information on this newsgroup is posted weekly - read before writing!
Contains links to    |  http://www.pedt.serve.net.uk/faq/clpb-faq.txt
helpful information  |  http://www.merlyn.demon.co.uk/clpb-faq.txt
and some guidelines  |  ftp://garbo.uwasa.fi/pc/doc-net/faqclpb.zip



Tue, 21 Oct 2003 16:34:44 GMT  
 How to do fast sequential binary read ?

Quote:
> You gave the answer yourself: Buffer.

Well, thats exactly what I ***DONT*** want to do. Is there
no way to use the builtin buffer of File ? Or isn't there
such a buffer ?!?!?


Tue, 21 Oct 2003 19:12:10 GMT  
 How to do fast sequential binary read ?

Quote:


>> You gave the answer yourself: Buffer.

> Well, thats exactly what I ***DONT*** want to do. Is there
> no way to use the builtin buffer of File ? Or isn't there
> such a buffer ?!?!?

There isn't.

File is an untyped filetype to build your own IO on top of.



Tue, 21 Oct 2003 19:37:36 GMT  
 How to do fast sequential binary read ?

Quote:

>> You gave the answer yourself: Buffer.

>Well, thats exactly what I ***DONT*** want to do. Is there
>no way to use the builtin buffer of File ? Or isn't there
>such a buffer ?!?!?

There isn't. You provide your own.
--
Information on this newsgroup is posted weekly - read before writing!
Contains links to    |  http://www.pedt.serve.net.uk/faq/clpb-faq.txt
helpful information  |  http://www.merlyn.demon.co.uk/clpb-faq.txt
and some guidelines  |  ftp://garbo.uwasa.fi/pc/doc-net/faqclpb.zip


Tue, 21 Oct 2003 20:01:25 GMT  
 How to do fast sequential binary read ?

Quote:



> >> You gave the answer yourself: Buffer.

> > Well, thats exactly what I ***DONT*** want to do. Is there
> > no way to use the builtin buffer of File ? Or isn't there
> > such a buffer ?!?!?

> There isn't.

> File is an untyped filetype to build your own IO on top of.

Ouch.

And the only 'advanced' file type seems to be 'Text' which
is already a text file ...

Okay, thank you.



Tue, 21 Oct 2003 20:33:42 GMT  
 How to do fast sequential binary read ?

Quote:




>> >> You gave the answer yourself: Buffer.

>> > Well, thats exactly what I ***DONT*** want to do. Is there
>> > no way to use the builtin buffer of File ? Or isn't there
>> > such a buffer ?!?!?

>> There isn't.

>> File is an untyped filetype to build your own IO on top of.

> Ouch.

> And the only 'advanced' file type seems to be 'Text' which
> is already a text file ...

Maybe  "file of char;".

But implementing this kind buffering on file of type is not hard as it seems,
and more importantly, it is very fast because it is not generic, but
specific for your application.



Tue, 21 Oct 2003 21:30:18 GMT  
 How to do fast sequential binary read ?

Quote:




> >> >> You gave the answer yourself: Buffer.

> >> > Well, thats exactly what I ***DONT*** want to do. Is there
> >> > no way to use the builtin buffer of File ? Or isn't there
> >> > such a buffer ?!?!?

> >> There isn't.

> >> File is an untyped filetype to build your own IO on top of.

> > Ouch.

> > And the only 'advanced' file type seems to be 'Text' which
> > is already a text file ...

> Maybe  "file of char;".

> But implementing this kind buffering on file of type is not hard as it seems,
> and more importantly, it is very fast because it is not generic, but
> specific for your application.

Well, of course its not hard to implement, but I hate reinventing
the wheel.


Wed, 22 Oct 2003 00:32:44 GMT  
 How to do fast sequential binary read ?

Quote:



>> Maybe  "file of char;".

>> But implementing this kind buffering on file of type is not hard as it seems,
>> and more importantly, it is very fast because it is not generic, but
>> specific for your application.

> Well, of course its not hard to implement, but I hate reinventing
> the wheel.

See it not as reinventing, but practicing with. You didn't invent the
bicycle either, but you still had to learn it :_)

Or use Delphi/FPC and use TBufStream.



Wed, 22 Oct 2003 01:29:40 GMT  
 
 [ 10 post ] 

 Relevant Pages 

1. Read and write sequential file

2. FAST binary to hex string conversion

3. reading binary with freepascal

4. problems reading binary file, and outputting to screen.

5. reading binary file and outputting to standard output (screen)

6. Binary field read/write

7. Can Excel read a Pascal binary file?

8. I need to read a binary file

9. Reading Pascal-generated PACKED binary w/ C program

10. Reading binary files in pascal

11. Reading binary files in PASCAL ??

12. problem reading 4-byte reals from binary file

 

 
Powered by phpBB® Forum Software