Uploading forms+files via Python script -- no luck 
Author Message
 Uploading forms+files via Python script -- no luck

I've written a script that's supposed to  perform an http POST
in order to upload a mix of fields and files to a webserver,
based on examples referred to in previous posts on this topic here.

To test this script, I wrote a simple python cgi script that receives
a form POST then prints out everything that showed up in FieldStorage.
The test-script shows the form values if I make a simple form
and post fields and/or files from Internet Explorer (or Netscape).

When I use my upload script to attempt to post, however, I
never see any cgi form contents.  :-(

Using netcat to catch the raw HTTP traffic, I captured both the
http upload output of IE and my script, but didn't see any
differences in formatting/MIME stuff that looked detrimental.

But, something's gotta be wrong, because Python (and presumably
other webservers -- the real point) doesn't think I've sent a form. :(

Many thanks to anyone who spots the problem!!


Karl Ulbrich

p.s. It would be great if urllib did this...  I'd offer to do it,
     but, well, it doesn't work!

............... THE RECEIVE SCRIPT (install as a CGI) ...............
# This script simply echos out FieldStorage contents for-debugging
# (this script works just fine)

import sys,os,cgi,glob,string
sys.stderr = sys.stdout
data = cgi.FieldStorage()

print "content-type: text/html\n"
print "<h1>Echo Form Contents</h1>"

print str(len(data.keys())) + " Keys found in data.keys(): "
for key in data.keys(): print key, '\t'

print "<table border=1>"
for item in dir(data):
        print "<tr>", "<td>", item, "</td>",
        print "<td>", getattr( data, item), "</td>", "</tr>"
print "</table>"

.............. THE (broken) UPLOAD SCRIPT .........................

# Summary: First stab at a general web form-and-file POST uploader

# Liberally borrows and addapts from:
# FileTransfer.py - CGI file transfer, may require Python 1.5 or higher
# Prior Author/Version:

# Prior art:  Aaron Watters, Jim Fulton

import os, sys, traceback
import httplib
#import tempfile
from whrandom import randint
from string import join
from time import time

class FileUploadRequest:
    """Client-side CGI form/file POST upload."""
    def __init__(self, uri, host, port):
        self.uri = uri
        self.host = host
        self.port = port
        self.queryString = None
        self.boundary= '%s%s_%s_%s' % \
                        ('-------', int(time()), os.getpid(), randint(1,10000))
        self.parts = []  ## collect each multipart as a single list-entry

    def loadfile(self, filename, fieldname, headerDict=None):
        """Add a file to the multipart upload."""
        f = open(filename, 'rb')
        data = f.read()
        hdr = []
        hdr.append( '%s' % self.boundary )
        hdr.append('Content-Disposition: form-data; name="%s"; filename="%s"' % (fieldname,filename))
        hdr.append('Content-Type: application/octet-stream')
#        hdr.append('Content-Length: %d' % len(data))
        # Add user-defined header values here, if provided.
        if type(headerDict) == type({}):
            for k in headerDict.keys():
                hdr.append("%s: %s", (k, headerDict[k]))
        part='%s\n\n%s' % ( join(hdr,'\n') , data )
        self.parts.extend( [part] )

    def loadfield(self, fieldname, data, headerDict=None, ):
        """Add a field to the multipart upload."""
        hdr = []
        hdr.append( '%s' % self.boundary )
        hdr.append('Content-Disposition: form-data; name="%s"' % fieldname )
        #hdr.append('Content-Length: %d' % len(data))
        # Add user-defined header values here, if provided.
        if type(headerDict) == type({}):
            for k in headerDict.keys():
                hdr.append("%s: %s", (k, headerDict[k]))
        part='%s\n\n%s' % ( join(hdr,'\n') , data )
        self.parts.extend( [part] )

    def request(self):
        query = '%s\n%s' % ( join(self.parts,'\n'), self.boundary )

        print "=====BODY DATA BEGIN (DEBUG)====="
        print query
        print "=====BODY DATA END (DEBUG)======"

        contentType = 'multipart/form-data; boundary=%s' % self.boundary
        contentLength = str(len(query))
        h = httplib.HTTP()
        h.connect(self.host, self.port)
        h.putrequest('POST', self.uri)
        h.putheader('Accept', '*/*')
        h.putheader('Proxy-Connection', 'Keep-Alive')
        h.putheader('User-Agent', 'PyFormsPoster/0.1')
        h.putheader('Content-Type', contentType)
        h.putheader('Content-Length', contentLength)
        rcode, rmsg, headers = h.getreply()
        response = h.getfile().read()
        print "Return Code: %s\n" % rcode
        #if rcode != 200:
        #    msg = "error: %s, %s\n%s %s" % (rcode, self.uri, rmsg, response)
        #    #raise FileUploadRequestException(msg)
        #    raise "FileUploadRequestException", msg
        return response

def testecho():
    usage = 'usage: cgiupload filename'
    if len(sys.argv) < 2:
        print usage
    filename = sys.argv[1]

    uri = '/cgi-bin/echoform.py'
    host = ''
    port = 80
    #port = 8888

    F = FileUploadRequest(uri, host, port)
    F.loadfield( "yourname" , "Arthur, of Camelot" )
    F.loadfield( "summary" , "Perhaps if we built a large wooden Badger..." )
    F.loadfield( "details" , "Run Away!" )
    F.loadfield( "submit" , "submit the page" )
    response = F.request()
    print response

if __name__ == '__main__':
    if os.environ.has_key('SERVER_PROTOCOL'):

........... A SIMPLE HTML FORM ................................
<h1>Sends to echoform.py on 80</h1>
<pre><form action="http://hostname:80/cgi-bin/echoform.py" method="POST" enctype="multipart/form-data">
Your Name <input type='text' name='yourname' size='35'>
Summary   <input type='text' name='summary' size='35'>
File #1   <input name="file1" type="file" size="35">
Details   <textarea name=details ROWS="4" COLS="30" wrap=virtual></textarea>
          <input name="submit" type="submit" value="Upload File(s)">

Mon, 06 Jan 2003 03:00:00 GMT  
 Uploading forms+files via Python script -- no luck
For anyone interested in the solution...  I screwed up something when
adapting my (working) starting point to the code I posted.  

The MIME boundary specified in the header starts with "-----" (this may be
an arbitrary number, but 5 works).  

However, boundaries in the BODY have to start with an extra 2 hyphens,
so they need to start with "-------".

Furthermore, the last boundary ends with an extra "--" in the working
code example I had.  

Strange but true MIME stuff.  I gave up on the RFC, so that's just a
what-worked-for-me solution, assumed to be basically correct.

Karl Ulbrich

Mon, 13 Jan 2003 03:00:00 GMT  
 [ 2 post ] 

 Relevant Pages 

1. upload file via form

2. upload file via web form (Re: to Trent Mick's posting)

3. upload file via web form

4. upload file via web form

5. File uploads via a python cgi-bin program

6. Problem with Python script for uploading files

7. image upload - files not viewable via ftp

8. Uploading a file via POST ?

9. Ideas on how to edit/synchronize MYSQL data via file upload/download

10. uploading binary files via cgi

11. need sample for file-upload via http with auth

12. file upload via http


Powered by phpBB® Forum Software