Help with binary files, read, and eof 
Author Message
 Help with binary files, read, and eof

I'm attemping to read binary files, and am running into issue with
respect to EOF.  Been through the FAQ and the wiki for info on using
binary files, but can't seem to find the problem.

Short description: I'm reading in a binary file, 255 bytes at a time.
For each 255 byte "packet", I prepend 5 bytes, and send the whole thing
out a filehandle configured for the serial port.

My problem is with EOF handling.  My files that I'm processing are
always an integer multiple off 255 bytes in length.  The current file
I'm using is 9945 bytes = 255 bytes * 39 "packets".

When running my script, however, it always outputs the first 39
packets, without a problem, but then one additional packet
of garbage data.  I've stared at my EOF handling for hours, and
cannot pick out the problem.

Here's the main code, snipped to as short as I can manage.

set count 0
if [ catch { open $filename r } fh1 ] {
   puts stderr "Cannot open $filename : $fh1"

Quote:
} else {

   fconfigure $fh1 -translation binary -encoding binary
   set noquit 1
   while { $noquit } {
     set read_bytes [ read $fh1 255 ]
     set read_bytes_length [ expr [ string length $read_bytes ] ]

     # Send out Header to serial port ($fh)
     puts -nonewline $fh \xa0
     puts -nonewline $fh \x1e
     puts -nonewline $fh \x56
     puts -nonewline $fh \x05
     puts -nonewline $fh \x00
     flush $fh

     # Send out data to serial port
     if { $read_bytes_length != 0 } {
       puts -nonewline $fh $read_bytes
       puts stderr "Packet $count. read_bytes_length=$read_bytes_length"
       incr count
     }
     flush $fh

     # Check for EOF
     if [ eof $fh1 ] {
       for { set i $read_bytes_length } { $i < 255 } { incr i } {
         puts $fh \x00  # If, for some reason filesize is not an
                       # integer number of packets, then pad zeros
       }
       set noquit 0
     } elseif { $read_bytes_length != 255 } {
       puts stderr "ERROR read_bytes_length!=255, and EOF NOT detected."
     }
   }
   close $fh1

Quote:
}

The script shows "Packet 39. read_bytes_length=255" being sent out.
Only packets 0 - 38 should be sent.  (I keep thinking it some basic
problem with counting from 0 / counting from 1, but I can't see it).

Any help appreciated.
Using Tcl v8.4.4

--
Mark Curry



Sat, 22 Jul 2006 09:12:32 GMT  
 Help with binary files, read, and eof

Quote:

> I'm attemping to read binary files, and am running into issue with respect to
> EOF.  Been through the FAQ and the wiki for info on using binary files, but
> can't seem to find the problem.

There are issues with detecting EOF that are definite FAQs, like you
don't get an EOF signal until you attempt to read past the end of the
file.   http://wiki.tcl.tk/eof

Quote:
> Short description: I'm reading in a binary file, 255 bytes at a time.

(odd size, rather than 256)

Quote:
> When running my script, however, it always outputs the first 39
> packets, without a problem, but then one additional packet
> of garbage data.  I've stared at my EOF handling for hours, and
> cannot pick out the problem.

You have a bunch of output:
       ...

Quote:
>      puts -nonewline $fh \x00
>      flush $fh

before you test for eof.




Sat, 22 Jul 2006 10:46:13 GMT  
 Help with binary files, read, and eof
Hi Mark,

the changes below should help.

Best regards

Ulrich

Quote:

> I'm attemping to read binary files, and am running into issue with
> respect to EOF.  Been through the FAQ and the wiki for info on using
> binary files, but can't seem to find the problem.

> Short description: I'm reading in a binary file, 255 bytes at a time.
> For each 255 byte "packet", I prepend 5 bytes, and send the whole thing
> out a filehandle configured for the serial port.

> My problem is with EOF handling.  My files that I'm processing are
> always an integer multiple off 255 bytes in length.  The current file
> I'm using is 9945 bytes = 255 bytes * 39 "packets".

> When running my script, however, it always outputs the first 39
> packets, without a problem, but then one additional packet
> of garbage data.  I've stared at my EOF handling for hours, and
> cannot pick out the problem.

> Here's the main code, snipped to as short as I can manage.

> set count 0
> if [ catch { open $filename r } fh1 ] {
>   puts stderr "Cannot open $filename : $fh1"
> } else {
>   fconfigure $fh1 -translation binary -encoding binary
>   set noquit 1
>   while { $noquit } {
>     set read_bytes [ read $fh1 255 ]

Insert your 'check for EOF' here

- Show quoted text -

Quote:
>     set read_bytes_length [ expr [ string length $read_bytes ] ]

>     # Send out Header to serial port ($fh)
>     puts -nonewline $fh \xa0
>     puts -nonewline $fh \x1e
>     puts -nonewline $fh \x56
>     puts -nonewline $fh \x05
>     puts -nonewline $fh \x00
>     flush $fh

>     # Send out data to serial port
>     if { $read_bytes_length != 0 } {
>       puts -nonewline $fh $read_bytes
>       puts stderr "Packet $count. read_bytes_length=$read_bytes_length"
>       incr count
>     }
>     flush $fh

>     # Check for EOF
>     if [ eof $fh1 ] {
>       for { set i $read_bytes_length } { $i < 255 } { incr i } {
>         puts $fh \x00  # If, for some reason filesize is not an
>                    # integer number of packets, then pad zeros
>       }
>       set noquit 0

continue

- Show quoted text -

Quote:
>     } elseif { $read_bytes_length != 255 } {
>       puts stderr "ERROR read_bytes_length!=255, and EOF NOT detected."
>     }
>   }
>   close $fh1
> }

> The script shows "Packet 39. read_bytes_length=255" being sent out. Only
> packets 0 - 38 should be sent.  (I keep thinking it some basic problem
> with counting from 0 / counting from 1, but I can't see it).

> Any help appreciated.
> Using Tcl v8.4.4



Sat, 22 Jul 2006 21:50:25 GMT  
 Help with binary files, read, and eof

Quote:


>>I'm attemping to read binary files, and am running into issue with respect to
>>EOF.  Been through the FAQ and the wiki for info on using binary files, but
>>can't seem to find the problem.

> There are issues with detecting EOF that are definite FAQs, like you
> don't get an EOF signal until you attempt to read past the end of the
> file.   http://wiki.tcl.tk/eof

>>Short description: I'm reading in a binary file, 255 bytes at a time.

> (odd size, rather than 256)

255 is correct.  It's a Reed-Solomon Codeword (255,223).

Quote:

>>When running my script, however, it always outputs the first 39
>>packets, without a problem, but then one additional packet
>>of garbage data.  I've stared at my EOF handling for hours, and
>>cannot pick out the problem.

> You have a bunch of output:
>        ...

>>     puts -nonewline $fh \x00
>>     flush $fh

> before you test for eof.

Well, I need to take care of special cases, when the file size is NOT
multiples of 255 bytes.  In this case I need to write out whatever PART
of the 255 bytes that was in the file, fol]owed by enough zeros to fill
out to 255.

You'll notice I have that output in a conditional - only output if
$read_bytes_length != 0.  Unfortunetly, to me it looks like TCL's
broken.  On the 40'th read, it DOES set EOF.  But it also fills
read_bytes with garbage, and sets the string length at whatever
was requested.  It looks like there's no way to handle partial
reads?  On EOF, I need to output whatever is valid, then pad the
rest.

Read those FAQ's but doesn't seem to apply.  What does TCL do
when you do the read with the bytecnt option, and that many bytes
is not available?

Do I need to stat the file before hand, grab the bytecnt of the file,
do the math beforehand to precalculate the EOF?  Ugly.

Thanks,

Mark

--
Mark Curry



Sun, 23 Jul 2006 00:49:45 GMT  
 Help with binary files, read, and eof
Hi Mark,

Depending on the size of the file, you may want to try reading in the whole
file as a single string and then parsing the string.  You consume more
memory that way, but you'll resolve your EOF problem.

Regards,
Aric



Sun, 23 Jul 2006 02:01:58 GMT  
 
 [ 5 post ] 

 Relevant Pages 

1. Reading a binary file / writing binary data to a file

2. Binary file contains "LF" at EOF

3. Reading binary files, unflatten binary string

4. HELP! read binary from file

5. Newbee needs help, reading a binary file.

6. help for reading a binary file

7. Help with binary file read

8. HELP: Reading a binary file

9. HELP! Read binary file

10. Help! Need to read a binary output file

11. Help : Reading MS Fortran binary files.....

12. binary file read - Help

 

 
Powered by phpBB® Forum Software