Tempfile.py destructor problem (python 1.4) 
Author Message
 Tempfile.py destructor problem (python 1.4)

Hi folks,

I'm running python 1.4 on solaris.  I wanted to create a Tempfile class
which would act like a writable file, but which would automatically delete
the temp file on disk when the object was destructed.

The problem was that the tempfile was not always being deleted.  Because...
- the destructor was not always being called because...
- the refcount on the last constructed Tempfile object was not dropping to
zero because...
- apparently 'self' for the constructor continues to exist even after the
constructor method finishes.  (Maybe???)  This only seems to hapen when I
call tempfile.mktemp() from the constructor.

The workaround seems to be to set self to 0 (or somesuch) at the end of the
ctor.

Comments/Questions
- This is a bug, right?
- Has it been fixed in python 1.5?
- Is my fix safe?

Source code is attached.  Diagnostic prints have been left in.  To see the
bug, comment out the 'self=0' in the ctor, then run python interactively
and try doing things like...
- t1=Tempfile()
  del t1;
- t1=Tempfile()
  t2=Tempfile()
  del t1
  del t2
- Notice that even when you quit out of python, the last constructed
Tempfile is not deleted :-(.

..Pat

# Tempfile.py by Pat Gratton

# Tempfile class: automatically deletes when finished

import tempfile, os

class Tempfile:
    def __init__(self):
        'Generate new tempfile and open for write.'
        self._path = tempfile.mktemp()
        self._file = open(self._path,'w')
        print 'inited',self._path
        #--Note python seems to have a bug in that the constructor 'self'
        #--continues to exist, thus preventing the refcount for the last
        #--constructed object from dropping to zero, thus preventing the
        #--destructor for that object from ever running.  So...
        self = 0

    def __del__(self):
        'Close and delete temp file'
        print 'deleting',self,self._path
        self.close()
        print 'closed',self,self._path
        os.unlink(self._path)
        print 'unlinked',self,self._path

    def close(self):
        'Close temp file'
        self._file.close()

    def file(self):
        'Return file object.'
        return self._file

    def flush(self):
        'Flush temp file'
        self._file.flush()

    def path(self):
        'Return path to file'
        return self._path

    def write(self,x):
        'Write to temp file'
        self._file.write(x)

--
*---------------------------------*------------------------------------*
| Pat Gratton        Nirvana Sig->|                                    |

| http://www.*-*-*.com/ ~gratton   |                                    |
*---------------------------------*------------------------------------*



Tue, 06 Jun 2000 03:00:00 GMT  
 Tempfile.py destructor problem (python 1.4)

More destructor weirdness...

So having kluged a solution the problem of Tempfiles not destructing, I
create my Gnuplot class (basically a pipe to a gnuplot process, plus a
routine to handle plotting data).  (Yes I know that there's a Gnuplot.py in
Hinsen's package, but it doesn't do quite what I want.)  Background: when
plot() is given a sequence to plot, it writes the data in a Tempfile object
for gnuplot to read.  The Tempfile object is saved in an Gnuplot object
list.  That way, the tempfile should stick around until the Gnuplot object
destructs.

Well, I have the object not destructing problem again (apparently due to
tempfile.mktemp again), but I 'del self' at end of the rountine that calls
'mktemp' and that fixes that problem, but now...

When I quit from an interactive session, the Gnuplot destructor gets called
and the Tempfiles *try* to destruct, but an (undetailed) exception get's
thrown, and the temporary files don't actually get removed.  Aaargghh!
Eventually, I find this magic fix: in the Gnuplot class module, change
'from Tempfile import Tempfile' to 'import Tempfile' (and change the
instantiations accordingly, of course).  Voila!  It finally works, but for
non-obvious reasons.

Bug?  Design flaw?

..Pat

--
*---------------------------------*------------------------------------*
| Pat Gratton        Nirvana Sig->|                                    |

| http://www.pobox.com/~gratton   |                                    |
*---------------------------------*------------------------------------*



Tue, 06 Jun 2000 03:00:00 GMT  
 Tempfile.py destructor problem (python 1.4)

Pat Gratton:

Quote:
> I'm running python 1.4 on solaris.  I wanted to create a Tempfile class
> which would act like a writable file, but which would automatically delete
> the temp file on disk when the object was destructed.

> The problem was that the tempfile was not always being deleted.  Because...
> - the destructor was not always being called because...
> - the refcount on the last constructed Tempfile object was not dropping to
> zero because...
> - apparently 'self' for the constructor continues to exist even after the
> constructor method finishes.  (Maybe???)  This only seems to hapen when I
> call tempfile.mktemp() from the constructor.

> The workaround seems to be to set self to 0 (or somesuch) at the end of the
> ctor.

> Comments/Questions
> - This is a bug, right?
> - Has it been fixed in python 1.5?
> - Is my fix safe?

Your problem is probably caused by a common pitfall with destructors
in Python -- some function you call does a try-except and as a side
effect, a reference to your stack frame remains accessible through the
traceback stored in sys.exc_traceback.

I'm not sure whether to call it a bug -- it's not a coding bug, it
could be called a design problem, or an unexpected interaction between
destructors and exception handling.

And yes, it has been fixed in 1.5 (by redesigning the exception
handing implementation).

I've never seen your fix (setting self=0 before returning), but it is
cute and works, so I say, go for it.  It shows you're a creative soul!

Note that in 1.5, you can use tempfile.TemporaryFile() which does what
your Tempfile class does.

In a followup, Pat continues:

Quote:
> When I quit from an interactive session, the Gnuplot destructor gets called
> and the Tempfiles *try* to destruct, but an (undetailed) exception get's
> thrown, and the temporary files don't actually get removed.  Aaargghh!
> Eventually, I find this magic fix: in the Gnuplot class module, change
> 'from Tempfile import Tempfile' to 'import Tempfile' (and change the
> instantiations accordingly, of course).  Voila!  It finally works, but for
> non-obvious reasons.

This one has to do with the order in which modules are destroyed.  It
is an unfortunate problem that hasn't been solved completely in 1.5
either.  The reason that your solution works is that it delays the
destruction of the Tempfile module until after the Gnuplot module is
destroyed.  When your __del__ gets called after the Tempfile module
has been destroyed (actually, its __dict__ has been cleared), what
happens is that your call to "os.unlink" fails because the global
variable "os" is deleted.

--Guido van Rossum (home page: http://www.python.org/~guido/)



Tue, 06 Jun 2000 03:00:00 GMT  
 Tempfile.py destructor problem (python 1.4)

Quote:

> Hi folks,

> I'm running python 1.4 on solaris.  I wanted to create a  Tempfile class
> which would act like a writable file, but which would automatically delete
> the temp file on disk when the object was destructed.

Python 1.5 includes just such an object in the tempfile module.

Quote:
> The problem was that the tempfile was not always being deleted.  Because...
> - the destructor was not always being called because...
> - the refcount on the last constructed Tempfile object was not dropping to
> zero because...
> - apparently 'self' for the constructor continues to exist even after the
> constructor method finishes.  (Maybe???)  This only seems to hapen when I
> call tempfile.mktemp() from the constructor.

> The workaround seems to be to set self to 0 (or somesuch) at the end of the
> ctor.

I'm surprized that this would make any difference.

Quote:

> Comments/Questions
> - This is a bug, right?

Not exactly.  In general, you cannot rely on destructors being
called, especially for global variables, especially if
you compiled Python with thread support.  

If Python is compiled with thread support, it makes no
attempt to destroy global variables on exit.

If Python is compiled without thread support, it tries and often fails
to call destructors.

Quote:
> - Has it been fixed in python 1.5?

It may be improved slightly. In particular, it may do a little
better at calling destructors on exit if not compiled with threads.

Quote:
> - Is my fix safe?

It does no harm, but I suspect that it's effect is accidental.

Quote:
> Source code is attached.  Diagnostic prints have been left in.  To see the
> bug, comment out the 'self=0' in the ctor, then run python interactively
> and try doing things like...
> - t1=Tempfile()
>   del t1;
> - t1=Tempfile()
>   t2=Tempfile()
>   del t1
>   del t2
> - Notice that even when you quit out of python, the last constructed
> Tempfile is not deleted :-(.

I predict that if you put your test in a test function
so that the Tempfile objects are non-global, you will
not have the problem.

Also, assuming that your program knows when it is
done with the file and calls the close method,
you can do the unlink in the close and have the
__del__ method call close.  This is what is done
in tempfile.TempFile.

Jim

--

Technical Director   (540) 371-6909              Python Powered!
Digital Creattions   http://www.digicool.com     http://www.python.org



Wed, 07 Jun 2000 03:00:00 GMT  
 
 [ 4 post ] 

 Relevant Pages 

1. Problems with Py.assert in jython2.1 and java 1.4

2. Version problem -- Python 1.4 V Python 1.5.1

3. Problem with file tranfer with ftplib module in Python 1.4

4. Problem with Python 1.4 and Access on Windows NT

5. linking problem with Thread Module in Python 1.4

6. Class pickling in Python 1.4 - problem / feature

7. Problems running Python Ver 1.4 code

8. Problem building python 1.4 with shared tk 4.1 on IRIX 5.3

9. Problems building tkinter with Python 1.4, Tcl 7.5, Tk 4.1

10. tkinter problems after switch to Python 1.4

11. Problems with installing python 1.4 on HP-UX 9.05

12. Strange problem while installing python 1.4 on SGI Irix 5.3

 

 
Powered by phpBB® Forum Software