CGI.pm Bug: file upload file size mismatch when file size is close to buffer size 
Author Message
 CGI.pm Bug: file upload file size mismatch when file size is close to buffer size

I had submitted this bug report to the author/maintainer over 1 month
ago and have not heard back from him.  It is probable that he never
received the email so I am posting the submission here in order to
ensure that this bug is exposed to those who care and hopefully to the
maintainer if he did not receive my email.

Doug Perham                                          o{..}o    

WorldGate Communications, Inc.                        (______)\
                                                      / \  / \  

Subject: bug: file upload file size mismatch when file size is close to buffer size

Date: 27 Sep 2002 15:55:34 -0400

There is a difference in the size and content of a file POSTED to the
CGI.pm when the size of the file is close to a multiple of the CGI.pm
internal buffer size.  Attached, find a program that exposes this
problem by POSTing files of size 4094, 4095, 4096, 4097, and 4098 bytes
and compare the size to the size after the POST-ing.  You'll see that
sizes of 4095 and 4096 fail. Here is the fix for CGI.pm v2.86

--- CGI.pm.2.86 Wed Sep 18 15:37:56 2002

     my $bytesToReturn;    
     if ($start > 0) {           # read up to the boundary
-    $bytesToReturn = $start > $bytes ? $bytes : $start;
+    $bytesToReturn = $start - 2 > $bytes ? $bytes : $start;
     } else {    # read the requested number of bytes
     # leave enough bytes in the buffer to allow us to read

     substr($self->{BUFFER},0,$bytesToReturn)='';

     # If we hit the boundary, remove the CRLF from the end.
-    return (($start > 0) && ($start <= $bytes))
+    return ($bytesToReturn==$start)
            ? substr($returnval,0,-2) : $returnval;
 }
 END_OF_FUNC

--
Doug Perham                                          o{..}o    

WorldGate Communications, Inc.                        (______)\
                                                      / \  / \  

Other Info
---------

    * the version of CGI.pm (perl -MCGI -e 'print $CGI::VERSION')

2.86

    * the version of Perl (perl -v)

This is perl, v5.6.0 built for i386-freebsd

    * the name and version of your Web server

Server version: Apache/1.3.26 (Unix)
Server built:   Jul  1 2002 11:32:52

    * the name and version of the operating system you are using

FreeBSD 4.7-RC #18

    * a short test script that reproduces the problem (30 lines or less)

----- begin -----
#!/usr/local/bin/perl
use CGI;



{
   fork() && sleep(1) && next;
   # -- sequence of characters
   my $msg = join('', map chop(), (1 .. $size));
   printf STDERR "\n\nBefore POST: %d bytes, cksum %d\n", length($msg), unpack('%32a*', $m
sg);

   open(FILE, "> $size.orig") || die($!);
   print FILE $msg;
   close FILE;

   # -- construct a file for the simulated posting
   my $sep = "____This is a test to save out a file of $size bytes_____";
   my $post_msg = << "EOM";
--$sep\r
Content-Disposition: form-data; name="File"; filename="$size.txt"\r
Content-Type: text/plain\r
Content-Transfer-Encoding: binary\r
\r
$msg\r
--$sep--\r
EOM

   close(STDIN);
   open(FILE, "> POST-$size.txt") || die($!);
   print FILE $post_msg;
   close(FILE);

   # -- a POST is read off STDIN, make our input POST.txt the STDIN
   open(STDIN, "POST-$size.txt") || die($!);

   # -- set up the environment
   $ENV{REQUEST_METHOD} = 'POST';
   $ENV{CONTENT_LENGTH} = length($post_msg);
   $ENV{CONTENT_TYPE}   = "multipart/form-data; boundary=$sep";

   my $cgi     = new CGI() || die($!);

   # -- get the posted data and save it out
   my $file    = $cgi->param('File');
   my $new_msg = join('', <$file>);
   close $file;
   open(FILE, "> $file") || die("$file: $!");
   print FILE $new_msg;
   close FILE;

   printf STDERR "After  POST: %d bytes, cksum %d\n", length($new_msg), unpack('%32a*', $n
ew_msg);
   if ( length($msg) != length($new_msg) )
   {
      print STDERR "the message size before and after the POST do not match.\n"
                  ."compare the files $size.orig and $file\n\n";
   }
   undef($file);
   undef($cgi);
   last;

Quote:
}

----- end ------



Mon, 11 Apr 2005 01:45:33 GMT  
 CGI.pm Bug: file upload file size mismatch when file size is close to buffer size

Quote:

> I had submitted this bug report to the author/maintainer over 1 month
> ago and have not heard back from him.  It is probable that he never
> received the email so I am posting the submission here in order to
> ensure that this bug is exposed to those who care and hopefully to the
> maintainer if he did not receive my email.

Please keep trying!  The behavior of CGI.pm on your code has changed from
2.86 to 2.89 (v2.89 of CGI.pm still doesn't handle 4095 right), which
indicates to me that somebody's trying to fix it.  The mfd parser in
CGI.pm is really grotesque, so you might need to be more patient with
the maintainer(s).

Have you considered contributing your code to CGI.pm's test suite?

--
"Rule 3.  Fancy algorithms are slow when n is small, and n is usually small.
Fancy algorithms have big constants. Until you know that n is frequently going
to be big, don't get fancy..."
                                              -- Rob Pike



Mon, 11 Apr 2005 04:56:44 GMT  
 CGI.pm Bug: file upload file size mismatch when file size is close to buffer size

Quote:


> > I had submitted this bug report to the author/maintainer over 1 month
> > ago and have not heard back from him.  It is probable that he never
> > received the email so I am posting the submission here in order to
> > ensure that this bug is exposed to those who care and hopefully to the
> > maintainer if he did not receive my email.

> Please keep trying!  The behavior of CGI.pm on your code has changed from
> 2.86 to 2.89 (v2.89 of CGI.pm still doesn't handle 4095 right), which
> indicates to me that somebody's trying to fix it.  The mfd parser in
> CGI.pm is really grotesque,

Agreed :)  Although, the module is eminent in its usefulnes.

Quote:
> so you might need to be more patient with
> the maintainer(s).

Yes, but I also provided the fix, which I have tested and it works.

Quote:

> Have you considered contributing your code to CGI.pm's test suite?

I have no idea how to go about that process, other than to submit
it to the maintainers and have them bless it.

Quote:
> --
> "Rule 3.  Fancy algorithms are slow when n is small, and n is usually small.
> Fancy algorithms have big constants. Until you know that n is frequently going
> to be big, don't get fancy..."
>                                               -- Rob Pike

--
Doug Perham                                          o{..}o    

WorldGate Communications, Inc.                        (______)\
                                                      / \  / \  


Mon, 11 Apr 2005 21:02:02 GMT  
 CGI.pm Bug: file upload file size mismatch when file size is close to buffer size

[...]

Quote:
> Yes, but I also provided the fix, which I have tested and it works.

Ahh- sorry.  Somehow I elided your patch while composing my followup.
Out of sight, out of mind :-(

Quote:

> > Have you considered contributing your code to CGI.pm's test suite?

> I have no idea how to go about that process, other than to submit
> it to the maintainers and have them bless it.

Here's what I suggest:

  1) get the source tree for v2.89 : % perl -MCPAN -e "get CGI"
  2) cd to the CGI.pm-2.89/t/ subdir and make a file called upload.t
  3) rewrite your upload tests, making them conform to the methodology
     used in the other *.t files. Incorporate many more sizes than
     just 409*.
  4) email the maintainers your upload.t file, as well as your tested
     patch against 2.89.

I realize that this is an awful lot of additional work, but consider-

  1) right now, you are probably the person that best understands the
     problem with the current parser,
  2) the CGI.pm maintainers no doubt experience fear and loathing while
     making *any* changes to the multipart/form-data parser, especially
     ones they may not fully comprehend,
  3) AFAICT there are currently *no* tests in CGI.pm that exercise the
     multipart/form-data parser.

IMO, the last point is really the most important one. If you get your
tests incorporated into the CGI distribution, that's far better than
just convincing them to adopt your patch.

HTH.
--
Joe Schaefer    "Not everything that counts can be counted, and not everything
                                 that can be counted counts."
                                               --Albert Einstein



Mon, 11 Apr 2005 23:44:18 GMT  
 
 [ 4 post ] 

 Relevant Pages 

1. finding file size before uploading a file

2. file upload, determining file type, size, format

3. Need to know how to somehow limit file sizes in file uploads with Perl

4. Need to know how to limit the file sizes in file uploads in Perl

5. upload file size zero in CGI

6. CGI.pl and file upload size restraint problem

7. Upload CGI:maximum file size?

8. Differences in the file size reported by Net::FTP->size and ls -l

9. checking uploaded GIF / JPEG file size

10. Need file upload to display date, size

11. Size limit when uploading file ?

12. file upload default max size?

 

 
Powered by phpBB® Forum Software