Problem running Python script as cron job 
Author Message
 Problem running Python script as cron job

Hi all,

I've got a python script that runs very nicely from the command line,
but seems to hang up and loaf around forever if I run it as a cron job.  
From the command line, I do:

     /path/to/statsthing -r

That works well.  The script prints many status messages to STDOUT as it
runs.  I want to preserve these messages when running the script
automatically, so I do this in the crontab:

     /path/to/statsthing -r > /usr/local/temp/.log

This is a script I rewrote from an old Perl version as a learning
exercise to teach myself Python.  The Perl version runs from Cron very
happily with that command.  The Python version runs to a point, then
simply stops, and I need to kill the processes.  The log contains some
data, but I'm pretty sure some of it is never being flushed to disk, so
I'm not sure exactly where it's halting.

Anybody know of any issues with running python scripts from Cron?

Thanks,

-Bill



Tue, 24 Sep 2002 03:00:00 GMT  
 Problem running Python script as cron job
[William R.{*filter*}son, on Fri, 07 Apr 2000]
:: Anybody know of any issues with running python scripts from Cron?

No. I've had multiple Python scripts running every one, two, five and
ten minutes around the clock for years on multiple servers without
problems.

I'd suspect you might have a bug lurking.



Tue, 24 Sep 2002 03:00:00 GMT  
 Problem running Python script as cron job

Hi Bill

Try stderr?
/path/prog > /path/log 2>&1

Maybe it spits some errors?

Per.



Tue, 24 Sep 2002 03:00:00 GMT  
 Problem running Python script as cron job


Quote:
> Hi Bill

> Try stderr?
> /path/prog > /path/log 2>&1

> Maybe it spits some errors?

Tried that, but I still wind up with an empty log.  Anyway to force it
to write immediately rather than buffering?

Thanks,

-Bill



Fri, 27 Sep 2002 03:00:00 GMT  
 Problem running Python script as cron job

Quote:

> [William R.{*filter*}son, on Fri, 07 Apr 2000]
> :: Anybody know of any issues with running python scripts from Cron?

> No. I've had multiple Python scripts running every one, two, five and
> ten minutes around the clock for years on multiple servers without
> problems.

> I'd suspect you might have a bug lurking.

I think so too, but I can't find it, because the script runs flawlessly
every time I run it by hand, and the buffer never flushes the output to
the log when it's run by Cron.

Blorg.

-Bill



Fri, 27 Sep 2002 03:00:00 GMT  
 Problem running Python script as cron job

Quote:


> > Hi Bill

> > Try stderr?
> > /path/prog > /path/log 2>&1

> > Maybe it spits some errors?

> Tried that, but I still wind up with an empty log.  Anyway to force it
> to write immediately rather than buffering?


Unknown option: --
usage: python [option] ... [-c cmd | file | -] [arg] ...
Options and arguments (and corresponding environment variables):
-d     : debug output from parser (also PYTHONDEBUG=x)
-i     : inspect interactively after running script, (also PYTHONINSPECT=x)
         and force prompts, even if stdin does not appear to be a terminal
-O     : optimize generated bytecode (a tad; also PYTHONOPTIMIZE=x)
-OO    : remove doc-strings in addition to the -O optimizations
-S     : don't imply 'import site' on initialization
-t     : issue warnings about inconsistent tab usage (-tt: issue errors)
-u     : unbuffered binary stdout and stderr (also PYTHONUNBUFFERED=x)
-v     : verbose (trace import statements) (also PYTHONVERBOSE=x)
-x     : skip first line of source, allowing use of non-Unix forms of #!cmd
-X     : disable class based built-in exceptions
-c cmd : program passed in as string (terminates option list)
file   : program read from script file
-      : program read from stdin (default; interactive mode if a tty)
arg ...: arguments passed to program in sys.argv[1:]
Other environment variables:
PYTHONSTARTUP: file executed on interactive startup (no default)
PYTHONPATH   : ':'-separated list of directories prefixed to the
               default module search path.  The result is sys.path.
PYTHONHOME   : alternate <prefix> directory (or <prefix>:<exec_prefix>).
               The default module search path uses <prefix>/python1.5.

so that'll be `python -u' then ...

Cheers,
M.

--
  ... but I guess there are some things that are so gross you just have
  to forget,  or it'll destroy something within you.  perl is the first
  such thing I have known.                 -- Erik Naggum, comp.lang.lisp



Fri, 27 Sep 2002 03:00:00 GMT  
 Problem running Python script as cron job
[William R.{*filter*}son, on Mon, 10 Apr 2000]

::
:: > [William R.{*filter*}son, on Fri, 07 Apr 2000]
:: > :: Anybody know of any issues with running python scripts from Cron?
:: >
:: > No. I've had multiple Python scripts running every one, two, five and
:: > ten minutes around the clock for years on multiple servers without
:: > problems.
:: >
:: > I'd suspect you might have a bug lurking.
::
:: I think so too, but I can't find it, because the script runs flawlessly
:: every time I run it by hand, and the buffer never flushes the output to
:: the log when it's run by Cron.

Are you "running by hand" as the same user as your script? Maybe you
have a permissions problem, assuming the script needs to write to a
file or whatever.



Fri, 27 Sep 2002 03:00:00 GMT  
 Problem running Python script as cron job
There are a couple of ways to force your output to be immediately
flushed.

An ugly solution is a sys.stdout.flush() after every print statement.

A less ugly solution is to reopen sys.stdout as soon as
your script starts up, and to turn off output buffering:
  sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)

I thought it used to be possible to do a setbuf on the file object,
provided you hadn't already written to it.  But this appears not to
be the case.

Another option is to change your cron job so it sets the
PYTHONUNBUFFERED environment variable before starting your program.
Something like this might work:

  /usr/bin/env PYTHONUNBUFFERED=1 /path/to/statsthing -r >/usr/local/tmp/.log

For that matter you could write a shell script wrapper which
set the environment variable, then invoke it from cron:

  #!/bin/sh
  PYTHONUNBUFFERED=1
  export PYTHONUNBUFFERED
  /path/to/statsthing -r >/usr/local/tmp/.log

Here's hoping this is more help than hindrance...

--
Mitch Chapman

Quote:



> > [William R.{*filter*}son, on Fri, 07 Apr 2000]
> > :: Anybody know of any issues with running python scripts from Cron?

> > No. I've had multiple Python scripts running every one, two, five and
> > ten minutes around the clock for years on multiple servers without
> > problems. [...]

> I think so too, but I can't find it, because the script runs flawlessly
> every time I run it by hand, and the buffer never flushes the output to
> the log when it's run by Cron.



Fri, 27 Sep 2002 03:00:00 GMT  
 Problem running Python script as cron job

Quote:

> :: > :: Anybody know of any issues with running python scripts from Cron?
> :: >
> :: > No. I've had multiple Python scripts running every one, two, five
> :: > and
> :: > ten minutes around the clock for years on multiple servers without
> :: > problems.
> :: >
> :: > I'd suspect you might have a bug lurking.
> ::
> :: I think so too, but I can't find it, because the script runs
> :: flawlessly
> :: every time I run it by hand, and the buffer never flushes the output
> :: to
> :: the log when it's run by Cron.

> Are you "running by hand" as the same user as your script? Maybe you
> have a permissions problem, assuming the script needs to write to a
> file or whatever.

In both cases, it's running as root.

Thanks,

-Bill



Fri, 27 Sep 2002 03:00:00 GMT  
 Problem running Python script as cron job

Quote:

> > > Try stderr?
> > > /path/prog > /path/log 2>&1

> > > Maybe it spits some errors?

> > Tried that, but I still wind up with an empty log.  Anyway to force it
> > to write immediately rather than buffering?


> Unknown option: --
> usage: python [option] ... [-c cmd | file | -] [arg] ...
> Options and arguments (and corresponding environment variables):
> -d     : debug output from parser (also PYTHONDEBUG=x)
> -i     : inspect interactively after running script, (also
> PYTHONINSPECT=x)
>          and force prompts, even if stdin does not appear to be a
>          terminal
> -O     : optimize generated bytecode (a tad; also PYTHONOPTIMIZE=x)
> -OO    : remove doc-strings in addition to the -O optimizations
> -S     : don't imply 'import site' on initialization
> -t     : issue warnings about inconsistent tab usage (-tt: issue errors)
> -u     : unbuffered binary stdout and stderr (also PYTHONUNBUFFERED=x)
> -v     : verbose (trace import statements) (also PYTHONVERBOSE=x)
> -x     : skip first line of source, allowing use of non-Unix forms of
> #!cmd
> -X     : disable class based built-in exceptions
> -c cmd : program passed in as string (terminates option list)
> file   : program read from script file
> -      : program read from stdin (default; interactive mode if a tty)
> arg ...: arguments passed to program in sys.argv[1:]
> Other environment variables:
> PYTHONSTARTUP: file executed on interactive startup (no default)
> PYTHONPATH   : ':'-separated list of directories prefixed to the
>                default module search path.  The result is sys.path.
> PYTHONHOME   : alternate <prefix> directory (or <prefix>:<exec_prefix>).
>                The default module search path uses <prefix>/python1.5.

> so that'll be `python -u' then ...

> Cheers,
> M.

Heh...dowh.  If you're ever in Seattle, I owe you a beer :).

-Bill



Fri, 27 Sep 2002 03:00:00 GMT  
 
 [ 10 post ] 

 Relevant Pages 

1. Problem Cron-Job using Python-Script

2. HELP: running tcl script as a cron job.

3. Still problems running script from cron

4. expect: problems running script from CRON

5. Problems running expect script in the background/from cron

6. problem running expect script via cron

7. Running a cron job in SFTP using Expect

8. trouble running an expect script in cron versus command line

9. trouble running an Expect script from cron

10. Running expect scripts from cron..?

11. Running EXPECT script through cron

12. cannot run expect script under cron

 

 
Powered by phpBB® Forum Software