Bug in CAtlHttpClientT::ReadChunkedBody() ? 
Author Message
 Bug in CAtlHttpClientT::ReadChunkedBody() ?

I've been considering using the CAtlHttpClient class for my application, and
installed tried out the HttpClient sample application.

I'm finding that when I click the 'navigate' button having set the url to a
server that sends back  chunked responses, the sample application does not
read the body of the response, and gets into an error state.  (Try out a url
such as www.yahoo.com for the sample app. The server returns a chunked
response).

I debugged it a little bit and it did seem that

CAtlHttpClientT<TSocketClass>::ReadChunkedBody()

was not parsing and using the size field in the chunked response. Any one
else run into this issue ?

Thanks for any pointers,
-Ravi



Fri, 29 Apr 2005 00:44:41 GMT  
 Bug in CAtlHttpClientT::ReadChunkedBody() ?
Ravi,
I have a look at CAtlHttpClientT<TSocketClass>::ReadChunkedBody() in
\vs.net\vc7\atlmfc\include\atlhttp.inl. It properly calls
get_chunked_size() / get_chunked_data() and i do not see any error there.

After that I did some test myself: setup IIS server (ref Q278998 HOWTO
Enable Chunked Transfer Encoding with IIS); build \Microsoft Platform
Sdk\Samples\Web\iis\isapi\extensions\Chunk sample; then run HttpClient
again the server using following URL:
http://localhost/chunk/ctetest.dll?file=/chunk/Ctetest.c+chunksize=100
And it works Here is the output from the HttpClient window:

--==[[ HEADER ]]==--
HTTP/1.1 200 OK
Server: Microsoft-IIS/5.1
Date: Mon, 11 Nov 2002 07:55:53 GMT
Transfer-encoding: chunked
Content-type: text/plain
--==[[ BODY ]]==--
/*++

Copyright (c) 1997 - 2000  Microsoft Corporation
.. omitted ...
    *buf = '\0';
    return TRUE;

Quote:
}

Then I did some debugging.  it also proves that ReadChunkedBody works as
exepcted.  therefore it may not be the problem of CAtlHttpClientT
implementation here.

thanks
wei

--
This posting is provided "AS IS" with no warranties, and confers no rights.
Got .Net?  http://www.gotdotnet.com



Fri, 29 Apr 2005 16:42:28 GMT  
 Bug in CAtlHttpClientT::ReadChunkedBody() ?

Thanks for testing it out, Wei.

I wonder if the bug is related to chunk sizes - i.e. if you point the
HttpClient sample app url to www.yahoo.com, for example, the chunk sizes are
28K or so. There seems to be in initial chunk buffer size of 2048 in
atlhttp.inl. ( #define CHUNK_BUFF_SIZE 2048).

Maybe the thing to do is to then try the sample app with larger chunk sizes
(say 28K) (or try out you url test with chunksize = 28000 or something large
like that).

Thanks again,
-Ravi



Quote:
> Ravi,
> I have a look at CAtlHttpClientT<TSocketClass>::ReadChunkedBody() in
> \vs.net\vc7\atlmfc\include\atlhttp.inl. It properly calls
> get_chunked_size() / get_chunked_data() and i do not see any error there.

> After that I did some test myself: setup IIS server (ref Q278998 HOWTO
> Enable Chunked Transfer Encoding with IIS); build \Microsoft Platform
> Sdk\Samples\Web\iis\isapi\extensions\Chunk sample; then run HttpClient
> again the server using following URL:
> http://localhost/chunk/ctetest.dll?file=/chunk/Ctetest.c+chunksize=100
> And it works Here is the output from the HttpClient window:

> --==[[ HEADER ]]==--
> HTTP/1.1 200 OK
> Server: Microsoft-IIS/5.1
> Date: Mon, 11 Nov 2002 07:55:53 GMT
> Transfer-encoding: chunked
> Content-type: text/plain
> --==[[ BODY ]]==--
> /*++

> Copyright (c) 1997 - 2000  Microsoft Corporation
> .. omitted ...
>     *buf = '\0';
>     return TRUE;
> }

> Then I did some debugging.  it also proves that ReadChunkedBody works as
> exepcted.  therefore it may not be the problem of CAtlHttpClientT
> implementation here.

> thanks
> wei

> --
> This posting is provided "AS IS" with no warranties, and confers no
rights.
> Got .Net?  http://www.gotdotnet.com



Sat, 30 Apr 2005 00:19:46 GMT  
 Bug in CAtlHttpClientT::ReadChunkedBody() ?
Ravi,
I test chunksize=30000 and still do not meet any problem.  as to
CHUNK_BUFF_SIZE, the comment says:

  // Keeps track of the allocated size of t_chunk_buffer.
  // This size will change if we need to read chunks bigger
  // than the currently allocated size of t_chunk_buffer.
  long nTChunkBuffSize = CHUNK_BUFF_SIZE;

it is not likely a problem too.

thanks
wei

--
This posting is provided "AS IS" with no warranties, and confers no rights.
Got .Net?  http://www.gotdotnet.com



Sat, 30 Apr 2005 12:36:12 GMT  
 Bug in CAtlHttpClientT::ReadChunkedBody() ?

Thanks for you patience Wie.

I decided to debug what the issue was with the chunked data from
www.yahoo.com.

What I found was the yahoo web server is inserting spaces (0x20) characters
between the chunk size hex value and the 0x0d0x0a (\r\n) characters.

I modified get_chunked_size() (see the code below) to ignore 'spaces'
following the end of the hex digits, and then everything worked fine.

The question is then if the chunked data is in violation of RFC2616.  I need
to look closer at the spec, but  I don't think LWS (line white space)
separating the hex size token from the rest of the message body is violating
the standard. So maybe worth making the atl code a bit defensive for this
situation ?...

-Ravi

template<class TSocketClass>

inline CAtlHttpClientT<TSocketClass>::CHUNK_LEX_RESULT
CAtlHttpClientT<TSocketClass>::get_chunked_size(char *&pBuffStart, char
*&pBuffEnd, long* pnChunkSize)

{

CHUNK_LEX_RESULT result = LEX_ERROR;

char *pStop = NULL;

if (pBuffStart >= pBuffEnd)

result = LEX_OUTOFDATA;

else

{

long nResult = strtoul(pBuffStart, &pStop, 16);

while (*pStop == ' ')  <=== (added)

     pStop++;    <=== (added)



Quote:
> Ravi,
> I test chunksize=30000 and still do not meet any problem.  as to
> CHUNK_BUFF_SIZE, the comment says:

>   // Keeps track of the allocated size of t_chunk_buffer.
>   // This size will change if we need to read chunks bigger
>   // than the currently allocated size of t_chunk_buffer.
>   long nTChunkBuffSize = CHUNK_BUFF_SIZE;

> it is not likely a problem too.

> thanks
> wei

> --
> This posting is provided "AS IS" with no warranties, and confers no
rights.
> Got .Net?  http://www.gotdotnet.com



Sat, 30 Apr 2005 15:59:04 GMT  
 Bug in CAtlHttpClientT::ReadChunkedBody() ?
Ravi,
It is good to know that you've got the root cause of the problem.  Actually
I've tried to connect www.yahoo.com before; however since I am behind the
corporation firewall, I cannot see the chunking at all (per rfc2616
sec19.4.6); that is why I turn to an isapi sample for testing.

I will convey your concern about the extra space character to our enginers
too since I am not familiar with this kind of details.

thanks
wei

--
This posting is provided "AS IS" with no warranties, and confers no rights.
Got .Net?  http://www.gotdotnet.com



Sat, 30 Apr 2005 21:34:39 GMT  
 Bug in CAtlHttpClientT::ReadChunkedBody() ?
I am also getting some strange behavior with CAtlHttpClientT when
downloading chunked data.  There are no errors reported when calling
Navigate() and the value returned by GetBodyLength() is correct.  But
the buffer returned by GetBody() is currupt.

I checked to see if there were any strange characters in the chunk
size and didn't find anything between the size and the \r\n.



Sun, 01 May 2005 01:38:14 GMT  
 Bug in CAtlHttpClientT::ReadChunkedBody() ?
I discovered the bug that was causing my problem.  It is in the ATL
code.

On line 1965 in file "atlhttp.inl" you must include the length of the
buffer when appending.  This is because the CAtlIsapiBuffer::Append
method uses strlen to get the buffer size if none is provided.  Of
course this is not suitable for binary data.

So the fix looks like this... (Line 1965 of atlhttp.inl)

if (!m_current.Append(( LPCSTR )result_buffer,
result_buffer.GetLength() ))

Another problem was found on line 1596 of the same file where memcpy
was used to do an overlapped memory move.  memmove should be used
instead.  memcpy may have been inappropriately used in other places
too.

Sam Thomsen



Sun, 01 May 2005 06:10:30 GMT  
 
 [ 8 post ] 

 Relevant Pages 

1. Another intermittent bug in CAtlHttpClientT and its resolution

2. CAtlHttpClientT bug N2

3. CAtlHttpClientT bug?

4. bufferoverrunbufferoverrunbufferoverrunbufferoverrunbufferoverrunbufferoverrunbufferoverrunbufferoverrunbufferoverrunbufferoverrunbufferoverrunbufferoverrunbufferoverrunbufferoverrunbufferoverrunbufferoverrunbufferoverrunbufferoverrunbufferoverrunbuf

5. CABLE BOX DESCRAMBLER PLANS ((LEGAL)),.,.,,.,,,,,,,,,,,,,,,.,.,.,,,,,,,,,,,,,,,,,,,,,,,,.,,,,,,,,.,.,,,,,,,,,,,.,,,,,,, 8077

6. yacc-a-dy YACC yacc, Help me HACK!

7. yacc-a-dy YACC yacc, Help me HACK!

8. System.ExecutionEngineException - framework bug?

9. CIsapiExtension::DispatchStencilCall bug?

10. Bug in COleControl::BoundPropertyChanged and CConnectionPoint::Unadvise in MFC 7.0 (Release)

11. Bug in oleconn.cpp: CConnectionPoint::XConnPt::Unadvise

12. Enumeration Bug!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

 

 
Powered by phpBB® Forum Software