threading: how to kill a (possibly blocked) thread? 
Author Message
 threading: how to kill a (possibly blocked) thread?

[python2.0/linux 2.2.16/i686]
I'm using the threading module in a multithreaded server, with
Socketserver's ThreadingTCPServer.  My handler thread spawns a
couple of other threads, each of which may block on wfile.write or
rfile.read, or RLock.acquire.
It appears that when the handler thread exits,
sometimes a child may hang around blocked forever.  I end up with
hundreds of hung threads and eventually may run out of processes.

I would like
to do an explicit 'kill' from the handler -- but there's no such
call available through the Threading API.  I notice an 'exit'
function in the 'thread' module that 'Threading' claims to be built on.

Is there a way to get a 'thread' reference from my Threading object
and call it's exit function?  Will this confuse Threading and break
other things?

Thanks,
        George
--
Your mouse has moved.
Windows NT must be restarted for the change to take effect.
Reboot now?  [OK]



Sun, 06 Jul 2003 03:24:20 GMT  
 threading: how to kill a (possibly blocked) thread?

Quote:

> Is there a way to get a 'thread' reference from my Threading object
> and call it's exit function?  Will this confuse Threading and break
> other things?

Unless the exit() is called from within the thread itself, it won't
terminate the thread, and no, I'm not aware of any way to externally
terminate a python thread - at least not cleanly or in a supported manner.

Terminating a thread externally (from another thread) is not something
supported in most threading systems/OSs.  Those that have had some
mechanism have, I believe, had trouble ensuring proper behavior.  Most
systems recover resources at the process level, so cleanup on a
per-thread basis is not reliable.

What you really need to do is ensure that your threads can accept an
external request to terminate themselves, and then code the threads
defensively to ensure they check for such requests at some periodic
interval.  It's not always the simplest approach, but it's one of the
fairly standard consequences of threaded development.

Python threads are pretty nice in that your class representing your
thread can provide some extra methods that are actually intended to be
called from some other thread in order to set the signaling, avoiding
some of the global flag objects that tend to get used in other
threading environments.

For example, when your thread starts up, it can create an event object
in the reset state and store it within the object instance.  It could
then provide a method that a main application thread could call to
request the thread to exit which in turn would set that internal event
object.  The method is running within the context of the main
application thread, but accessing shared instance data which will then
be checked by the separate thread running that object.  The main
thread code should then check the event object periodically and exit
nicely if it is set.

This does mean that you have to keep the thread from blocking
indefinitely.  It doesn't mean you can't block at all, depending on
how much latency you are willing to put up with should you wish to
exit, or if the blocking is related to multi-thread interaction.  For
example, if you know that you may be blocked in an acquire() call,
your design may assure you of eventually getting control back as the
other threads begin to terminate (and given that at least one thread
must have the lock, and presumably is checking its exit flag while not
blocked).  With I/O, you should operate in a non-blocking or
interruptible mode (using select() for sockets, or files under Unix,
etc...) where you can get a periodic chance to check for the
termination request.

--
-- David
--
/-----------------------------------------------------------------------\

  |             FitLinxx, Inc.            \  Phone: (203) 708-5192    |
 /  860 C{*filter*}Street, Stamford, CT  06902   \  Fax: (203) 316-5150     \
\-----------------------------------------------------------------------/



Sun, 06 Jul 2003 12:19:09 GMT  
 threading: how to kill a (possibly blocked) thread?


Quote:
> What you really need to do is ensure that your threads
> can accept an external request to terminate themselves ..

The ideal way to do this is with a timed interrupt that
invokes a function in the thread. The thread would start
by setting this interrupt:

  set_interrupt(8, funcCallBack)

Then, funcCallBack gets invoked every 8 seconds,
interrupting what the thread is doing, and returning
to the interrupted code when the function returns.
Unfortunately, I don't see such a facility in the
library.

Quote:
> .. The main thread code should then check the event
> object periodically and exit nicely if it is set.

Except for certain kinds of processing, such as FSMs,
this gets REAL ugly REAL fast. It also fails to protect
against the very things for which you want a guard,
i.e., external events that never occur.

Russell

Sent via Deja.com
http://www.deja.com/



Sun, 06 Jul 2003 23:44:14 GMT  
 threading: how to kill a (possibly blocked) thread?



|> What you really need to do is ensure that your threads
|> can accept an external request to terminate themselves ..
|
| The ideal way to do this is with a timed interrupt that
| invokes a function in the thread. The thread would start
| by setting this interrupt:
|
|   set_interrupt(8, funcCallBack)
|
| Then, funcCallBack gets invoked every 8 seconds,
| interrupting what the thread is doing, and returning
| to the interrupted code when the function returns.
| Unfortunately, I don't see such a facility in the
| library.
|
|> .. The main thread code should then check the event
|> object periodically and exit nicely if it is set.
|
| Except for certain kinds of processing, such as FSMs,
| this gets REAL ugly REAL fast. It also fails to protect
| against the very things for which you want a guard,
| i.e., external events that never occur.

For me, thread dispatch from an event loop is really the only
way to go in a seriously multi-threaded system.  I do my
multithreaded programs on BeOS, where that event loop concept
is built into an OOP user interface system, and just about
anything is a thread.  (I'm writing this in a Python newsreader
where I count 14 threads at the moment, most of them waiting
patiently for a message in their event loop queue - mostly
windows, and a couple of network clients.)

It is far from ugly, really somewhat elegant.  Can't imagine
doing it any other way.

I would guess the reason you don't see an interrupt feature
would be that there is nothing portable in that department,
and if there were, it would probably be a pain to integrate
into Python.  One general portability problem is that the
interrupt will often cost you the blocking I/O that was
interrupted;  then you have to trap the exception raised by
the interrupted I/O call, and post the I/O again.

I take no position on killing threads, in principal, but
since you can't do it from Python anyway, threads have to
be more cooperative, and the stuff that happens has to be
better understood and accounted for.




Mon, 07 Jul 2003 02:06:51 GMT  
 threading: how to kill a (possibly blocked) thread?

Quote:

>> What you really need to do is ensure that your threads
>> can accept an external request to terminate themselves ..

>The ideal way to do this is with a timed interrupt that
>invokes a function in the thread. The thread would start
>by setting this interrupt:

>  set_interrupt(8, funcCallBack)

Where does set_interrupt come from?

I can't find it in any of the documentation.  I did a search on
www.python.org, and it claims a hit on section 4 (Exception
Handling) of the API reference manual, but when examined, it
doesn't contain the string "set_interrupt".

A search of comp.lang.python turns up only the occurrance quoted
above.

--
Grant Edwards                   grante             Yow!  AIEEEEE! I am having
                                  at               an UNDULATING EXPERIENCE!
                               visi.com            



Mon, 07 Jul 2003 02:28:58 GMT  
 threading: how to kill a (possibly blocked) thread?


Quote:
> Where does set_interrupt come from?

You didn't read my post. The second sentence after:
"Unfortunately, I don't see such a facility in the
library." I was describing what I want, not what is
there. I share your disappointment.

Russell

Sent via Deja.com
http://www.deja.com/



Mon, 07 Jul 2003 03:27:12 GMT  
 threading: how to kill a (possibly blocked) thread?


Quote:
> For me, thread dispatch from an event loop is really
> the only way to go in a seriously multi-threaded system. ..

Yep. That works great, with trusted code. You know
the event responses are short, and return, so there
is no need for the threads to do anything except
respond to events in their queue. Put in a "stop"
event, and it will do what you want. The problem is:
What if you don't fully trust the code? See my post
on guarding execution time.

Russell

Sent via Deja.com
http://www.deja.com/



Mon, 07 Jul 2003 03:31:08 GMT  
 threading: how to kill a (possibly blocked) thread?

Quote:

>> Where does set_interrupt come from?

>You didn't read my post. The second sentence after:
>"Unfortunately, I don't see such a facility in the
>library." I was describing what I want, not what is
>there. I share your disappointment.

Doh!  I'd been worrying about that particular problem for a
while and I guess I got too e{*filter*}d when I saw the first part
of the post and just plain missed that sentence.

--
Grant Edwards                   grante             Yow!  My FAVORITE group
                                  at               is "QUESTION MARK & THE
                               visi.com            MYSTERIANS"...



Mon, 07 Jul 2003 06:06:26 GMT  
 threading: how to kill a (possibly blocked) thread?

Quote:

>   set_interrupt(8, funcCallBack)

> Then, funcCallBack gets invoked every 8 seconds,
> interrupting what the thread is doing, and returning
> to the interrupted code when the function returns.
> Unfortunately, I don't see such a facility in the
> library.

The issue is most likely that there's nothing I'm aware of in that
vein that is at all portable - also the ability to interrupt a pending
activity can be problematic.  Under Unix for example, not all I/O
calls are interruptable and they also don't all restart nicely even if
you may have SA_RESTART on the signal in question.  Then there's the
whole delivery of signals/interrupts to other than the main thread
problem on many systems.

I don't disagree with the sentiment, and have found myself wishing I
could just block in an operation in a thread more than once - just
trying to explain why it likely isn't in Python itself, nor likely to
be in the near future.

But with that said, I've got plenty of multi-threaded code that does
just fine with the "don't ever really block indefinitely" model, in
terms of addressing the original poster's question.

Quote:
> > .. The main thread code should then check the event
> > object periodically and exit nicely if it is set.

> Except for certain kinds of processing, such as FSMs,
> this gets REAL ugly REAL fast. It also fails to protect
> against the very things for which you want a guard,
> i.e., external events that never occur.

I'd need a specific example, but I'd think an FSM would be ideal for
this model, since by its nature you have some primary loop which is
making the state automaton decisions, and thus you should have a key
opportunity to deduce that it's time to exit.  And if you're just
sitting in a state without anything to do, well spend the time blocked
on some event that can be woken up should an exit request come in.

I'm not sure I follow the last sentence because I'm referring to
coding so that you never block indefinitely, but instead use whatever
mechanisms may be available to wait for the event you want but in some
timed manner so you get the occasion to check for an exit every once
in a while.  And that's specifically to guard against the external
event never occurring.

Typically the most troubling can be I/O operations that don't support
any way to timeout or monitor the status without blocking.  Often,
I've found that this is because the portable approach is more limited
than the native approach, which can be discouraging since most other
aspects of Python threading are so clean and portable.

--
-- David
--
/-----------------------------------------------------------------------\

  |             FitLinxx, Inc.            \  Phone: (203) 708-5192    |
 /  860 C{*filter*}Street, Stamford, CT  06902   \  Fax: (203) 316-5150     \
\-----------------------------------------------------------------------/



Mon, 07 Jul 2003 13:18:03 GMT  
 
 [ 9 post ] 

 Relevant Pages 

1. Thread#kill doesn't kill processes inside a thread

2. How to kill a thread from another thread ?

3. threads killing other threads

4. How to kill I/O blocked thread?

5. Threads creating threads creating threads...

6. thread, threading, mutex modules and non-threading interpreters

7. Possibly OT: Waiting for threads to complete...

8. thread mystery on Solaris -- possibly explained

9. How can you kill a thread ???

10. Vispro Rexx can't kill a rxftp thread

11. Is there a REXX Kill Thread Function?

12. killing a thread and its child

 

 
Powered by phpBB® Forum Software