eof on channel, not eof on transform: [eof] returns true 
Author Message
 eof on channel, not eof on transform: [eof] returns true

Hi,

  I have a problem  with a transformation that buffers data:
when Tcl_Eof()  returns true for the  underlying channel but
data  is  available  from  the  internal  buffer,  my  input
function  returns  a  chunk  of  data from  the  buffer  and
correctly returns the chunk size.

  When  data in  the internal  context is  processed  it can
explode in an unpredictable amount of bytes, maybe enough to
satisfy  the request;  however I  do not  want to  flush the
internal  context before  querying  the underlying  channel,
because this is  inefficient. It is not my  case but one can
imagine a transform that does not allow flushing the context
until the end of the stream.

  The TCL code in a test case that triggers the error is:

    while { ! [eof $channel] } {
        append data [read $channel 4096]
    }
    # here data is still in the transform internal context

    append data [read $channel]
    close $channel

this   code  reads   all   the  data   correctly  from   the
channel+transformation, so there exists a solution.

  However: the  [read] out of  the loop should not  be there
because  [eof] is  supposed  to return  true  only when  the
transformation has signaled the EOF condition; inspection of
Tcl_StackChannel() (TCL  version 8.4.5) shows  that actually
the transformation and its underlying channel share the same
state structure (of type  ChannelState): this leads to [eof]
returning true when the underlying channel is in EOF but the
transformation is not.  Hence the need for another [read].

  The  doc  about  channels  at SourceForge  states  that  a
transformation should not return the EOF condition until its
internal  buffers are  flushed:  this is  what my  transform
does, but it does not make any difference.

  I  think that this  is incorrect  behaviour: at  least the
[eof] man page should describe this.

  Hypothesis: a possible minimal effort solution could be to
clear the  EOF state from the ChannelState  structure if the
input function  returns enough  data to satisfy  the request
(in  GetInput() and  Tcl_ReadRaw(), generic/tclIO.c);  so if
the underlying  channel-or-transform sets the  EOF the above
transformation can clear it.

  So can someone suggest me what to do?

1. File a bug report for the [eof] commmand.

2. File a bug report for Tcl_Eof().

3. File a bug report for the [eof] manual page.

4. Take more time to study the TCL code because I am missing
   something.

5. Other.

--
Ciao,
  Marco Maggi



Sun, 08 Oct 2006 03:35:03 GMT  
 eof on channel, not eof on transform: [eof] returns true

Quote:

>   However: the  [read] out of  the loop should not  be there
> because  [eof] is  supposed  to return  true  only when  the
> transformation has signaled the EOF condition; inspection of
> Tcl_StackChannel() (TCL  version 8.4.5) shows  that actually
> the transformation and its underlying channel share the same
> state structure (of type  ChannelState): this leads to [eof]
> returning true when the underlying channel is in EOF but the
> transformation is not.  Hence the need for another [read].

It looks like the transformation is your own code, so why don't
you just clear the flag in appropriate places and set it, when
your buffers are done?

To me it looks like this sharing is a convenience for those
who stack something more trivial on top of a channel, so in
most cases they need not care about setting this flag, however
in your case you seem to be responsible for maintaining it
your appropriate way.

PS: I've never done any channel-stuff on C-level, so what I
  wrote here could be complete bu^H^H nonsense.



Sun, 08 Oct 2006 15:48:41 GMT  
 eof on channel, not eof on transform: [eof] returns true

Quote:

>...  so why  don't you  just clear  the flag  in appropriate
>places and set it, when your buffers are done?

This   requires   accessing   a   TCL   private   structure.
"ChannelState"   declaration   is   not  reachable   through
"tclInt.h", as far as I  can see. I'm not exactly sure about
how the state  is managed (there is a "sticky"  EOF and so I
guess  that there  is also  a  non-sticky one,  I think  for
non-blocking short reads).

Quote:
>To me it looks like  this sharing is a convenience for those
>who stack something more trivial  on top of a channel, so in
>most cases they need not care about setting this flag...

Yes. Probably only weird transformations are subject to this
problem.   My case  is  unlikely to  be  useful: a  readable
transformation  that compresses  (rather  than decompresses)
data.

I see that  I'm breaking the model of  the transformation: I
do  not flush  the  internal context  to  its output  buffer
before   checking   if  there   is   enough   data  in   the
transformation itself to  satisfy the request.  But flushing
the context for each [read] makes the compression useless.

If it works on 98% of the cases it is not a bug... :)

--
Ciao,
  Marco Maggi



Mon, 09 Oct 2006 04:35:52 GMT  
 
 [ 3 post ] 

 Relevant Pages 

1. EditableMovie.EOF goes 1 frame past EOF?

2. eof-error-p/eof-value

3. file.read(num) returns empty string before EOF.

4. How do I avoid a read from stdin returning nothing on EOF

5. early eof returned by spawned processes in expect on MP Solaris boxes

6. vwait fileevent on files returns eof constantly

7. encountering EOF not at the End Of File

8. while not eof(1): Translate Please...

9. Socket Infinitely Readable, Not EOF

10. solaris f90 not catching EOF errors?

11. Q: expect: expect eof not enough.

12. EOF in regexp: how to address?

 

 
Powered by phpBB® Forum Software