python, shell, environment variable 
Author Message
 python, shell, environment variable

hi, folks,

I am trying to get a set of python and bash scripts to work, and I
constantly run into trouble with environment variables:

First, my python script need to modify environment variable, say,
PATH, and want to keep the modification even after the script is done.
os.environ["PATH"]= ... doesn't seem to work, any idea?

Second, it would be better if my python script can call bash shell and
still keep the environment variable modification done by bash. I tried
os.popen("source some_shell"), it doesn't work. ?

Thanks for help.

-oliver



Sat, 27 Nov 2004 14:19:29 GMT  
 python, shell, environment variable

Quote:

> hi, folks,

> I am trying to get a set of python and bash scripts to work, and I
> constantly run into trouble with environment variables:

> First, my python script need to modify environment variable, say,
> PATH, and want to keep the modification even after the script is done.
> os.environ["PATH"]= ... doesn't seem to work, any idea?

Modifying environment variables this way is simply impossible: a process
run by another process cannot change the environment of its parent, so your
Python script will not be able to change environment variables in a
persistent way.

Quote:
> Second, it would be better if my python script can call bash shell and
> still keep the environment variable modification done by bash. I tried
> os.popen("source some_shell"), it doesn't work. ?

Same answer: this is impossible. os.popen runs the command in a subshell,
and the environment variable changes made by the "source" will only be
visible in that sub-shell.

One way to achieve what you want to do is to describe the environment
variables you want to change in a file created by the sub-process. That
file will then be used by the parent to do the actual variable settings.
Here is an example:

---------foo.sh---------------------
cat > /tmp/envvars <<.
MYVAR1=value1
MYVAR2=value2
.
------------------------------------

---------foo.py---------------------
import os
os.system('foo.sh')
for l in open('/tmp/envvars').readlines():
  var, val = l.strip().split('=')
  os.environ[var] = val
# ...
print os.environ['MYVAR1']
------------------------------------

This works from a bash child to a Python parent. The reverse is also
possible using the same mechanism: the Python script may simply create a
file containing "setenv" commands that will be source'd by the parent bash
script.

HTH
--

PragmaDev : Real Time Software Development Tools - http://www.pragmadev.com



Sat, 27 Nov 2004 18:15:28 GMT  
 python, shell, environment variable

Quote:

> hi, folks,
> I am trying to get a set of python and bash scripts to work, and I
> constantly run into trouble with environment variables:
> First, my python script need to modify environment variable, say,
> PATH, and want to keep the modification even after the script is done.
> os.environ["PATH"]= ... doesn't seem to work, any idea?

 Yes.  You seem to have a fundamental misunderstanding about how
 environment variables work (under UNIX and UNIX-like OSes, at least).

 The environment is a region of your process' memory that is not
 overwritten by an exec*() system call.  Thus it is an efficient way
 for a parent process to pass some (textual or textually encoded) data
 to its children.  The environment is *NOT* a share memory mechanism.

 To execute Python your shell performed a fork() and then an exec*()
 (some form of exec; it's not important which one).  fork() creates a
 new private address space for your new process, and address space that
 is initially an (almost) exact copy of the parent's memory (except for
 the portion of the stack or heap that's holding the return value to
 the fork() call itself).  [Please understand that this is a new
 "virtual" address space; the actual physical memory pages are usually
 shared through page table aliasing on modern processors/architectures
 running modern kernels --- using a technique called CoW: copy on write].

 So your assignments to os.environ["PATH"]=something are just happening
 to your address space.  They can't affect your parent's memory (that
 would be a SEGV, segmentation violation; or it would require some form
 of shared memory).  The assignments would be visible in any processes
 you created *under* your python process (os.popen('printenv') and search
 for the PATH entry for one way to demonstrate that), and they should
 remain set in that process (that was python) if you finish your script
 with one of the os.exec*() functions (except for the exec*e() which
 require you to explicitly prepare and pass a new environment to them).

Quote:
> Second, it would be better if my python script can call bash shell and
> still keep the environment variable modification done by bash. I tried
> os.popen("source some_shell"), it doesn't work. ?

 os.popen is creating yet another process; and the source command
 therein is modifying popen's memory.

Quote:
> Thanks for help.
> -oliver

 This is a fundamental consequence of UNIX' fork()/exec*() and
 memory management/protection model.  It is a basic principle that
 leads to several FAQs.  It is something I have to emphasize at
 considerable length in my shell scripting classes.  It is also one
 of the best examples of how important it is to understand the big
 picture, the "model" of an architecture, an OS, a language, or any
 other technology that one must control and use in any but the most
 trivial ways.  Once you understand how fork(), exec*() and *NIX
 memory management works then you'd never have this question --- because
 the answer would be so obvious.  (When my son tattoos his arm, why
 can't my grandpa read it off *his* arm?  Because it's a different skin
 that's being modified!).


Sat, 27 Nov 2004 17:28:58 GMT  
 
 [ 3 post ] 

 Relevant Pages 

1. Read Shell Environment Variables (Strings) into Verilog-XL?

2. Changing environment variables into python variables

3. Help on reading shell variable/environment

4. awk variables --> shell variables

5. Shell environment for system() call

6. Fuzzy Logic Environment Shell

7. Setting environment for current shell

8. Environment for shell scripts

9. Environment of shell called from tcl?

10. Tcl and shell environment

11. problems with shell environment in os.system calls

12. run a python application inside the python shell

 

 
Powered by phpBB® Forum Software