dbi pg_dbd performance 
Author Message
 dbi pg_dbd performance

Has anybody else found that the ruby dbi + pg_dbd performance to be slow?

I'm running a unit test that selects all the records in a single
PostgreSQL table. I'm finding that the select of about 300 records
takes about 0.2sec using the direct ruby postgres binding while the
DBI takes anywhere from 2.8-6 seconds.

--
Alan Chen
Digikata LLC
http://www.*-*-*.com/



Mon, 08 Nov 2004 09:38:29 GMT  
 dbi pg_dbd performance
Hello,

Unfortunately I discovered that ruby standard de{*filter*} is somehow too
slow to be usable but in simple cases. I suspect that there are ways
to make it much faster with some C code that would take over the current
set_trace_proc mechanism:

A) An extended caller() with bindings in the result
B) set_trace_proc( some_file, some_line_no ) { gets here when line reached }

I don't do C code anymore unfortunately (well... I do Ruby code instead)
Anyone working on this ?

Yours,

Jean-Hugues

-------------------------------------------------------------------------
Web:   http://www.*-*-*.com/
Phone: +33 (0) 4 92 27 74 17



Tue, 09 Nov 2004 00:52:46 GMT  
 dbi pg_dbd performance

Quote:

> Unfortunately I discovered that ruby standard de{*filter*} is somehow too
> slow to be usable but in simple cases. I suspect that there are ways
> to make it much faster with some C code that would take over the current
> set_trace_proc mechanism:

> A) An extended caller() with bindings in the result
> B) set_trace_proc( some_file, some_line_no ) { gets here when line reached }

I think it could be a lot faster if call_trace_func() were replaced with
a function pointer.  This would allow a trace func to be implemented
easily in C (instead of the way it is now, where the call still has to
be a Ruby function).  By default, that function pointer could point to
call_trace_func.

I think where much of the speed hit comes from is in the many many calls
to rb_f_binding.  The de{*filter*} appears to make use of the binding, so
the trick is to reduce the number of bindings created.  There are only a
few cases where the binding is actually needed.  The binding can be
created either by calling rb_f_binding() (which would need to be exposed
by making it non-static) or by calling the original call_trace_func
(which would also need to be non-static).

A C replacement for call_trace_func must also be VERY careful to not
call any ruby functions (particularly functions that might raise
exceptions).  From what I can tell, the current call_trace_func goes to
a lot of trouble to make sure that it cleans up after itself.
(Solution: if you want to make calls into ruby, do it by first calling
the original call_trace_func and having it do the actual call).

There's probably a lot more to consider, and a lot that can go wrong,
and a little more of the Ruby interpreter needs to be exposed to the C
extension.  I'm not sure whether or not the time required to put into
speeding up the de{*filter*} would by the time gained by having a faster
de{*filter*}.

Paul



Tue, 09 Nov 2004 02:12:00 GMT  
 dbi pg_dbd performance

Quote:
> Has anybody else found that the ruby dbi + pg_dbd performance to be slow?

> I'm running a unit test that selects all the records in a single
> PostgreSQL table. I'm finding that the select of about 300 records
> takes about 0.2sec using the direct ruby postgres binding while the
> DBI takes anywhere from 2.8-6 seconds.

What version of DBI are you using?  Can you upgrade?  There have been
major speed improvements in DBD::Pg recently that have significantly
reduced the time it takes to perform operations that return a row
(factor of a thousand or so).  -sc

--
Sean Chittenden



Tue, 09 Nov 2004 06:27:22 GMT  
 dbi pg_dbd performance

Quote:
> Unfortunately I discovered that ruby standard de{*filter*} is somehow
> too slow to be usable but in simple cases. I suspect that there are
> ways to make it much faster with some C code that would take over
> the current set_trace_proc mechanism:

> A) An extended caller() with bindings in the result
> B) set_trace_proc( some_file, some_line_no ) { gets here when line reached }

> I don't do C code anymore unfortunately (well... I do Ruby code
> instead) Anyone working on this ?

Have you tried rbprof?  It's a significantly faster profiler that's
available via Aspect Orriented programming.  -sc

--
Sean Chittenden



Tue, 09 Nov 2004 06:32:35 GMT  
 dbi pg_dbd performance

Quote:

> > Has anybody else found that the ruby dbi + pg_dbd performance to be slow?

> > I'm running a unit test that selects all the records in a single
> > PostgreSQL table. I'm finding that the select of about 300 records
> > takes about 0.2sec using the direct ruby postgres binding while the
> > DBI takes anywhere from 2.8-6 seconds.

> What version of DBI are you using?  Can you upgrade?  There have been
> major speed improvements in DBD::Pg recently that have significantly
> reduced the time it takes to perform operations that return a row
> (factor of a thousand or so).  -sc

I thought I did, rechecked it and found out that I had installed DBI
0.15 to the wrong location. (I wish the dbi package would default to
the /usr/local tree). The unit test and query code has been changed
somewhat, so now I'm getting

~0.06 sec - postgres binding
~0.14 sec - DBI binding

Which is still a little more than a factor two difference, but into
the range where I can justify trading off DBI for postgres.

- alan

--
Alan Chen
Digikata LLC
http://digikata.com



Wed, 10 Nov 2004 00:40:26 GMT  
 
 [ 10 post ] 

 Relevant Pages 

1. Renamed DBI to Ruby/DBI

2. malloc performance affecting python performance

3. ST80 performance; STA performance and booleans

4. dbi -> sqlrelay -> mssql performance

5. [VSE, Parts DBI Wrapper]

6. output DBI::Utils::TableFormatter.ascii to something other than STDOUT

7. ruby dbi obdc driver to remote postgresl (broken)?

8. Ruby DBI, MySQL and Win2K Server

9. DBI/OCI8 & binary data

10. DBI and MSAccess

11. Problem with timeout for DBI on Solaris

12. Why doesn't the dbi install work?

 

 
Powered by phpBB® Forum Software