headless application sometimes stuck hogging CPU 
Author Message
 headless application sometimes stuck hogging CPU

We have a headless VW3.0 application running on Solaris 2.6 that does some
processing of files and exits.  Once in a GREAT while, it has never
exited, and is seen consuming tons of CPU time.  Then:

a. You try again with the same initial conditions and it works great.

b. Or, it happens again, but works fine for someone else given the exact
same set of files.

I could imagine there being some NFS problems or DNS problems or ATM card
problems, etc.  But whatever hardware problems there are, they don't keep
happening.  It's as if a hardware glitch may be throwing the application
or VisualWorks off the rails and it's doomed to never complete.

Any theories, or thoughts on how to debug this?

By the way, this is with the 3.1b engine.
--
Wayne Johnston



Wed, 18 Jun 1902 08:00:00 GMT  
 headless application sometimes stuck hogging CPU
If you suspect the problems are outside the Smalltalk vm (which
it certainly sounds like it is), then you might try running the
application under 'truss'.  The man page gives details, but you
should be able to just do something like
        truss vw visual.im

If your application does its thing and then hangs, you can at least
see what system calls it is looping on.  That may get you closer
to the problem.

-Christopher

Quote:

> We have a headless VW3.0 application running on Solaris 2.6 that does some
> processing of files and exits.  Once in a GREAT while, it has never
> exited, and is seen consuming tons of CPU time.  Then:

> a. You try again with the same initial conditions and it works great.

> b. Or, it happens again, but works fine for someone else given the exact
> same set of files.

> I could imagine there being some NFS problems or DNS problems or ATM card
> problems, etc.  But whatever hardware problems there are, they don't keep
> happening.  It's as if a hardware glitch may be throwing the application
> or VisualWorks off the rails and it's doomed to never complete.

> Any theories, or thoughts on how to debug this?

> By the way, this is with the 3.1b engine.
> --
> Wayne Johnston



Wed, 18 Jun 1902 08:00:00 GMT  
 headless application sometimes stuck hogging CPU

Quote:

> If you suspect the problems are outside the Smalltalk vm (which
> it certainly sounds like it is), then you might try running the
> application under 'truss'.

Thanks, Christopher.

"VW Solution 1310: ParcTip 130: Unix system hang diagnostic procedure" is
also interesting.  It mentions the "trace" command which we don't have on
Solaris.  Also, there is a local policy that we cannot routinely use
"truss" due to its resource needs.  One hopeful lead is to kill the
process to get a core file and see what a de{*filter*} says was happening.

Is it possible to write code in the image to do something when it gets a
UNIX signal?  For debugging it would be valuable to just dump the stack
and exit.
--
Wayne Johnston



Wed, 18 Jun 1902 08:00:00 GMT  
 headless application sometimes stuck hogging CPU
Search cls for a post by Eliot Miranda that describes one way to rig up
image-level signal handlers for Unix signals.

pete

Quote:


> > If you suspect the problems are outside the Smalltalk vm (which
> > it certainly sounds like it is), then you might try running the
> > application under 'truss'.

> Thanks, Christopher.

> "VW Solution 1310: ParcTip 130: Unix system hang diagnostic procedure" is
> also interesting.  It mentions the "trace" command which we don't have on
> Solaris.  Also, there is a local policy that we cannot routinely use
> "truss" due to its resource needs.  One hopeful lead is to kill the
> process to get a core file and see what a de{*filter*} says was happening.

> Is it possible to write code in the image to do something when it gets a
> UNIX signal?  For debugging it would be valuable to just dump the stack
> and exit.
> --
> Wayne Johnston



Wed, 18 Jun 1902 08:00:00 GMT  
 headless application sometimes stuck hogging CPU

Quote:

> Search cls for a post by Eliot Miranda that describes one way to rig up
> image-level signal handlers for Unix signals.

Perhaps create a high priority Smalltalk process that blocks on reading
a named pipe. Any byte stuffed into that pipe will unblock the process
which can then make your diagnostic snapshot.

Reinout
-------



Wed, 18 Jun 1902 08:00:00 GMT  
 headless application sometimes stuck hogging CPU

Quote:

> Search cls for a post by Eliot Miranda that describes one way to rig up
> image-level signal handlers for Unix signals.

Thanks.  Eliot posted an article, and I believe it was he who copied to
<http://wiki.cs.uiuc.edu/VisualWorks/Unix+Signal+Handling>.  But is there
a way to do it without DLLCC, as we don't have it?

Wayne Johnston



Wed, 18 Jun 1902 08:00:00 GMT  
 headless application sometimes stuck hogging CPU
We've routinely kept some sort of log file for our batch programs.  for
debugging, one can at least send periodic notes to the transcript to
help close in on the problem area. If you are having memory problems,
this information may help: "ObjectMemory
growthDuringCurrentSession printString"


Wed, 18 Jun 1902 08:00:00 GMT  
 headless application sometimes stuck hogging CPU

Quote:

> Perhaps create a high priority Smalltalk process that blocks on reading
> a named pipe. Any byte stuffed into that pipe will unblock the process
> which can then make your diagnostic snapshot.

Great idea.  That's what I ended up doing:

        UnixProcess cshOne: 'mknod pipe p'.
        pipeAccessor := UnixPipeAccessor openFileReadOnly: 'pipe'.
        callingProcess := Processor activeProcess.

        [pipeAccessor readWait.
        HeadlessImage default
                dumpStackToTranscript: callingProcess suspendedContext]
                        forkAt: Processor timingPriority

Christopher Painter-Wakefield had suggested "truss".  I prematurely
rejected that idea, not realizing that you can attach to a current process
to see what it's doing.  So I'll try that next time.  I will also kill the
process via "kill -3" to get a core file to examine.
--
Wayne Johnston



Wed, 18 Jun 1902 08:00:00 GMT  
 
 [ 8 post ] 

 Relevant Pages 

1. Pausing execution without hogging the CPU?

2. CFD3.xxx on Windows NT hog cpu

3. S87 app is hogging the CPU, what is best fix

4. CPU hogging

5. CPU Hogging in Win NT/2000

6. CPU Hogging

7. CPU Hogging Under NT4

8. CPU Hog and FT_IAmIdle (revisited)

9. Need help - CPU Hog (only partially successful so far)

10. idle cpu hog fixed but patcha no longer works

11. What REXX commands are CPU hogs ?

12. Local CPU hogging upto 25% due to wish task

 

 
Powered by phpBB® Forum Software