Libraries implementation in C or in Python? Evolution of Python, Jython... 
Author Message
 Libraries implementation in C or in Python? Evolution of Python, Jython...

Are Python's libraries generally written in C or in Python, and why?
Someone commented on another thread that the re.search() function  ran
faster in its 1.5.2 version than in its 2.1 version, because in 1.5.2 it
was implemented in C and in 2.1 it was implemented in Python. It was
suggested to instead use the "pre" module, but I see it is old and may
become obsolete? I am curious as to the reasoning behind why these kinds
of changes and decisions are made?

In general, as python evolves from version to version, is Python code
likely to get slightly faster in execution time, or slightly slower?
Mind you, I am a Pythonista because of the numerous other beauties of
the language, not because of any speed issues; and I can certainly see
why Python should be and is optimized for other criteria, not speed, and
if this sometimes means a little step backwards in speed of code
execution then so be it. But I am just curious about the trade-offs, and
why such decisions are made.

There have been a few threads about 2.x versions of Python being a
little slower, in terms of the execution speed of similar code, than
1.5.2. Is this speed decrement likely to be permanent? Can anyone
comment on this?

I hesitate to ask any of this, because my extreme state of newbieness
and lack of knowledge do not give me any right to even speak up and ask
such questions. I only rush in where I should fear to tread because the
tangible beauty of the evolutionary process of a work of art such as the
Python programming language e{*filter*}s my curiosity and sense of wonder.

And then there is Jython. Is Python the mother and Java the father, or
is it the other way around? Does anyone else see the evolution of
Python, Jython and future offspring as a living work of art?

Ron Stephens

"...And from the heights of Mount Olympus, the mighty Patriarch C looks
on with pride as His progeny mutate under the new arts of cryo-genetic
surgery, His chromosomes are mixed-in, multiply inherited, and
ultimately transcended..." quoted from the Ars Cryogenia, Class
Phylogeny, opus five.



Fri, 28 Nov 2003 09:47:07 GMT  
 Libraries implementation in C or in Python? Evolution of Python, Jython...
[Ron Stephens]

Quote:
> Are Python's libraries generally written in C or in Python,

Both; e.g., look in your Lib/ directory and you'll find a couple megabytes
of Python library source.

Quote:
> and why?

Complex coding is easier in Python; low-level fiddling is easier and runs
faster in C.

Quote:
> Someone commented on another thread that the re.search() function  ran
> faster in its 1.5.2 version than in its 2.1 version, because in 1.5.2 it
> was implemented in C and in 2.1 it was implemented in Python. It was
> suggested to instead use the "pre" module, but I see it is old and may
> become obsolete? I am curious as to the reasoning behind why these kinds
> of changes and decisions are made?

sre was primarily motivated by the new need to support Unicode (pre couldn't
handle it).  Fredrik also thought he could write a matching engine faster
than pre, and turns out he was right.  But any project has to be completed
in finite time, and there can't be a guarantee that every case will be no
slower than before.  In the way I use regexps, I almost always find that sre
is faster.

There are no "general reasons" here:  each thing has its own reasons.

Quote:
> In general, as Python evolves from version to version, is Python code
> likely to get slightly faster in execution time, or slightly slower?

Neither, really, although the trend since 1.5.2 has been to get a bit slower
each time.  Two things account for that:

1. New features can cost.  For example, rich comparisons added many
   more possibilities for the runtime code to sort through, and *simple*
   comparisons got significantly slower as a more-than-less direct
   result.  Or cyclic gc removes worries about trash cycles leaking,
   but it takes time to find those cycles (before they simply leaked).

2. Fixing ancient bugs.  For example, it's possible to crash any Python
   through 2.1 by filling a dict with class instance keys, where the
   class overrides comparison with a function that mutates the dict
   while it's comparing.   This isn't sane, but it is possible.  Plugging
   this hole required additional code in the dict lookup routines, and
   that slows them down.  Plugging holes often costs a little, and then
   plugging many holes adds up to more than a little.

OTOH, current CVS Python is significantly quicker than 2.1 by several
accounts, as people have found 2.1 hotspots and added patches to cool them
off.  On the third hand, Python's speed is also significantly affected by
the compiler and OS; for example, on identical hardware, pystone runs
significantly faster under Win2K+MSVC than under Linux+gcc, but test_bufio
runs 10x faster under Linux; etc.  "The speed" of a x-platform app is a
slippery concept, because it's not independent of platform.

Quote:
> Mind you, I am a Pythonista because of the numerous other beauties of
> the language, not because of any speed issues; and I can certainly see
> why Python should be and is optimized for other criteria, not speed, and
> if this sometimes means a little step backwards in speed of code
> execution then so be it. But I am just curious about the trade-offs, and
> why such decisions are made.

Decisions to accept or reject a new feature don't generally take speed into
account, because it's generally impossible to predict in advance.  What
happens instead is that if an accepted feature is "too slow", someone who
cares enough will speed it over the next release cycle.  It's actually rare
that anyone cares enough to do that, though.

Quote:
> There have been a few threads about 2.x versions of Python being a
> little slower, in terms of the execution speed of similar code, than
> 1.5.2. Is this speed decrement likely to be permanent?

It's impossible to say without specific code to time on a specific platform.
The only way you'll ever get a clear answer to something as vague as
"similar code is a little slower -- permanent?" is if a developer chimes in
with "Doh! I *knew* I shouldn't have left sleep(1) in the inner loop"
<wink>.

Quote:
> ...
> I hesitate to ask any of this, because my extreme state of newbieness
> and lack of knowledge do not give me any right to even speak up and ask
> such questions. I only rush in where I should fear to tread because the
> tangible beauty of the evolutionary process of a work of art such as the
> Python programming language e{*filter*}s my curiosity and sense of wonder.

Rest assured (or appalled <wink>) that Python is pretty much the same as any
large project in this respect:  the speed freaks worry about speed, the
feature freaks push for new features, the good-old-days freaks complain
about change, and nobody has enough time to do all they want to do.

Quote:
> And then there is Jython. Is Python the mother and Java the father, or
> is it the other way around?

They're more like {*filter*}age siblings, who can't believe their parents ever had
sex so each suspects the other of being adopted.

Quote:
> Does anyone else see the evolution of Python, Jython and future
> offspring as a living work of art?

Undoubtedly.  But they usually keep it to themselves <wink>.


Fri, 28 Nov 2003 11:12:29 GMT  
 
 [ 2 post ] 

 Relevant Pages 

1. ANN: disipyl - a 2D and 3D plotting library for Python/Jython

2. ANN: disipyl - a 2D and 3D Python and Jython plotting library

3. ANN: disipyl - a 2D and 3D Python and Jython plotting library

4. ANN: disipyl - a 2D and 3D plotting library for Python/Jython

5. Jython and Python Libraries

6. Python 2.1 == Jython 2.1 != Python 2.2?

7. small python implementation or python subset ?

8. failed assert when importing a boost.python-created library from an embedded python interpreter

9. Using dynamic libraries with Python C extensions in embedded Python apps fails

10. Python Un*x build with a Python shared library

11. The Evolution Of Python

12. Calling Fortran .so/.dll from jython or python!

 

 
Powered by phpBB® Forum Software