SysCreateObject gets slower and slower 
Author Message
 SysCreateObject gets slower and slower

Platform: eCS preview 1

I've written a Rexx script that has to create a lot of objects.
(It scans the entire disk for documentation files.)  For the first
couple of hundred objects it works just fine, but as it gets
further on in the job it gets slower and slower.  I've gotten it
to the point where it's taking 15 seconds per object on a
400 MHz processor, and during that time the processor
utilisation is pegged at 100%.  In fact it must be getting worse
than that, because I've also had it to the point where it's
been running for half an hour, hogging the processor but
apparently making no progress at all.  By tracing the program
I've found that the long delay is in SysCreateObject -- everything
else in the script runs at normal speed.

One way to make the program run much faster is to open the
folder (on the desktop) in which it's creating the objects.  I can't
see why this would make a difference, but it does.  But this just
delays the inevitable, the program just gets a bit further before
slowing down.

I've tried this in both classic Rexx and object Rexx, and the
behaviour is the same in both cases.  If this were a compiled
programming language then I'd be suspecting something like
running out of file handles, but as far as I know that's not an
issue in Rexx.

Can anyone guess what might be going wrong?  Has anyone
had the same experience?

--

See http://www.*-*-*.com/ for OS/2 information and software



Sat, 07 Feb 2004 13:17:53 GMT  
 SysCreateObject gets slower and slower

Quote:

> I've written a Rexx script that has to create a lot of objects.
> (It scans the entire disk for documentation files.)  For the first
> couple of hundred objects it works just fine, but as it gets
> further on in the job it gets slower and slower.

I've done something similar, and experienced that handling a lot of
objects was indeed a resource hog. I did not get such extremely slow
results as you, but that's probably because I split the created
objects on a lot of target folders.

Regards, Johnnie



Sat, 07 Feb 2004 14:57:51 GMT  
 SysCreateObject gets slower and slower
Hello Peter,


(include some samples and a more specific description (Operating System,
version of REXX / Object REXX, ...)).

We try as soon as possible to understand and solve the problem.

Thank you very much.

--
|- Dr. Lavrentios Servissoglou
|- Project Lead, REXX / Object REXX Development
|- IBM Deutschland Entwicklung GmbH
|- phone: +49 7031 164417



Quote:
> Platform: eCS preview 1

> I've written a Rexx script that has to create a lot of objects.
> (It scans the entire disk for documentation files.)  For the first
> couple of hundred objects it works just fine, but as it gets
> further on in the job it gets slower and slower.  I've gotten it
> to the point where it's taking 15 seconds per object on a
> 400 MHz processor, and during that time the processor
> utilisation is pegged at 100%.  In fact it must be getting worse
> than that, because I've also had it to the point where it's
> been running for half an hour, hogging the processor but
> apparently making no progress at all.  By tracing the program
> I've found that the long delay is in SysCreateObject -- everything
> else in the script runs at normal speed.

> One way to make the program run much faster is to open the
> folder (on the desktop) in which it's creating the objects.  I can't
> see why this would make a difference, but it does.  But this just
> delays the inevitable, the program just gets a bit further before
> slowing down.

> I've tried this in both classic Rexx and object Rexx, and the
> behaviour is the same in both cases.  If this were a compiled
> programming language then I'd be suspecting something like
> running out of file handles, but as far as I know that's not an
> issue in Rexx.

> Can anyone guess what might be going wrong?  Has anyone
> had the same experience?

> --

> See http://eepjm.newcastle.edu.au for OS/2 information and software



Sat, 07 Feb 2004 18:02:03 GMT  
 SysCreateObject gets slower and slower

Quote:

>Hello Peter,

>(include some samples and a more specific description (Operating System,
>version of REXX / Object REXX, ...)).
>We try as soon as possible to understand and solve the problem.
>Thank you very much.

I don't think this problem is related to classic or object rexx. I think the problem
is within the WPS itself. You can try it with a C compiled emx program to do the
same and you'll get the same resoults.

ofcourse i could be wrong. but my experience shows its a wps problem.

--
(To email me, replace the .com with .gr)
t H.I.C. & D.B.S. t OS/2 Warp t Hellas t
t ServerConfig t ConfigEdit t OS/2 UK UG t
t Live fast, die young, and leave a handsome corpse. t



Sat, 07 Feb 2004 21:28:36 GMT  
 SysCreateObject gets slower and slower
Both WPS and the Container Control have massive scalability problems with
large numbers of objects. These are due to algorithmic "design decisions";
there is no known resolution.


Quote:
> Platform: eCS preview 1

> I've written a Rexx script that has to create a lot of objects.
> (It scans the entire disk for documentation files.)  For the first
> couple of hundred objects it works just fine, but as it gets
> further on in the job it gets slower and slower.  I've gotten it
> to the point where it's taking 15 seconds per object on a
> 400 MHz processor, and during that time the processor
> utilisation is pegged at 100%.  In fact it must be getting worse
> than that, because I've also had it to the point where it's
> been running for half an hour, hogging the processor but
> apparently making no progress at all.  By tracing the program
> I've found that the long delay is in SysCreateObject -- everything
> else in the script runs at normal speed.

> One way to make the program run much faster is to open the
> folder (on the desktop) in which it's creating the objects.  I can't
> see why this would make a difference, but it does.  But this just
> delays the inevitable, the program just gets a bit further before
> slowing down.

> I've tried this in both classic Rexx and object Rexx, and the
> behaviour is the same in both cases.  If this were a compiled
> programming language then I'd be suspecting something like
> running out of file handles, but as far as I know that's not an
> issue in Rexx.

> Can anyone guess what might be going wrong?  Has anyone
> had the same experience?

> --

> See http://eepjm.newcastle.edu.au for OS/2 information and software



Sun, 08 Feb 2004 13:44:32 GMT  
 SysCreateObject gets slower and slower

Quote:


>(include some samples and a more specific description (Operating System,
>version of REXX / Object REXX, ...)).

Thanks for the offer, and I'll certainly send an example program if
you think it will help.  However, the other messages in this thread
have convinced me that the problem is in the WPS rather than in
Rexx, so there's probably nothing you could do about it.

--

See http://eepjm.newcastle.edu.au for OS/2 information and software



Sun, 08 Feb 2004 14:34:15 GMT  
 SysCreateObject gets slower and slower

Quote:

>Both WPS and the Container Control have massive scalability problems with
>large numbers of objects. These are due to algorithmic "design decisions";
>there is no known resolution.

OK, you and Dimitris have succeeded in convincing me that this is
a fundamental WPS problem.   In fact I started suspecting the WPS
after realising that the program ran a whole lot faster if I opened
the desktop folder that was being populated.  (A totally
counterintuitive result, because with the folder open the WPS has
more work to do.)

Does Rexx have any way to close the folder again?  Maybe I
need to switch back to object Rexx to achieve that.

I'm still going to try the trick of putting my objects into many folders,
and then moving them at the end into the intended folder.  (In
fact I planned to do that this morning, and then other jobs came
up ...).  But I suspect that it won't work, because presumably I'm
going to get that same slowdown when doing the final move.

--

See http://eepjm.newcastle.edu.au for OS/2 information and software



Sun, 08 Feb 2004 14:40:24 GMT  
 SysCreateObject gets slower and slower

Quote:


>> I've written a Rexx script that has to create a lot of objects.
>> (It scans the entire disk for documentation files.)  For the first
>> couple of hundred objects it works just fine, but as it gets
>> further on in the job it gets slower and slower.

>I've done something similar, and experienced that handling a lot of
>objects was indeed a resource hog. I did not get such extremely slow
>results as you, but that's probably because I split the created
>objects on a lot of target folders.

Hey, that's a really useful comment.  Maybe I could create lots of
target folders, and then consolidate the results at the end.  I'll
give that a try tomorrow.  Thank you.

--

http://eepjm.newcastle.edu.au



Sun, 08 Feb 2004 18:54:29 GMT  
 SysCreateObject gets slower and slower
On Tue, 21 Aug 2001 10:02:03, "Lavrentios Servissoglou"

Quote:

> Hello Peter,


> (include some samples and a more specific description (Operating System,
> version of REXX / Object REXX, ...)).

> We try as soon as possible to understand and solve the problem.

Can other ORexx problems be reported to this e-mail too?
There is very {*filter*} semaphore leak in every version of OS/2 OREXX.
Every RexxStart creates a number of semaphores. They are freed when
program exits, but process that calls RexxStart in circle (i.e. for
hooks) will either fail or even make the whole system unstable
(because there are no free semaphores).
I've tried reporting this using general IBM process and got usual "Not
known problem", "Pay to fix" answer.

--
Bye,
        Vit

P.S. Use tivv at ukr.net address to reply.



Mon, 09 Feb 2004 22:57:57 GMT  
 SysCreateObject gets slower and slower

Quote:

> Platform: eCS preview 1

> I've written a Rexx script that has to create a lot of objects.
> (It scans the entire disk for documentation files.)  For the first
> couple of hundred objects it works just fine, but as it gets
> further on in the job it gets slower and slower.

That is the WPS itself that times you out.

There are 3 ways to go around:

Way number one:
        - create some folders and fill only 100 objects in each
                But at least you has significantly to do more
                work if you'll all objects into the same folder.
                You has to move each object from each folder
                into its last target and destroy
                the help folders. This creates a lot of
                IDs inside OS2.INI who never gets removed.
Way number two:
        - create some douzend objects - then sleep a while
          then create the nect couple of objects
          The number of objects createable in a loop without sleep
          shrinks with each object that gets created in a single folder
        - If there are many hundret or thousend objects to create it
          would cost long, long time to get ready.
Way number tree:
        - write a WPS class, that does the whole work.
        - Use your REXX procedure ony to resiter it,
          create a instance of it, let it do the whole work,
        - destroy the object, deregiste it's class
        - You need knowledge about SOM and WPS programming for that!
          But you'll done all your runtime work in seconds.

It's the WPS itself that goes crazy if too much objects created at
once. Letting the procedure get sleeping in intervals will help the
WPS to get enough CPU to do all what it has to do. If not, it caches
some work - and every time this cache is full it can't receive new
orders, so the process that sends this order has to wait unil a free
place in the cache is found. To find one cost time and more time.....

So sending some orders, then sleeping, then sending some others,
then.....

helps the WPS its asyncronous work done in time.

SysCreateObject is a wrapper to WinCreateObject what is a wrapper to
WPCreateObject.
WPCreateObject is a side door into the WPS. This side door needs a lot
of inerprocess communication, so it's one of the neadle ears on that
way.

If you have a WPS object, the time needed to create a single object is
significantly reduced, the code needed too. And the needed
communication between the WPS objects is inside the WPS very short,
outside very, very long.

--
Tschau/Bye
        Herbert

               Member 53 of Team OS/2 Germany
FIDONET : 2:2476/493          Maillbox : 49-7273-93072 analog + ISDN
http://www.dv-rosenau.de/     home of SQED/32: http://www.sqed.de

OS/2 is the more effective way to utilize your computer.



Thu, 12 Feb 2004 00:55:36 GMT  
 SysCreateObject gets slower and slower
Hello Vit,

yes, every ORexx (for Windows, for AIX, for OS/2, for Linux) problem may be
reported to this e-mail address.

--
|- Dr. Lavrentios Servissoglou
|- Project Lead, REXX / Object REXX Development
|- IBM Deutschland Entwicklung GmbH
|- phone: +49 7031 164417



Quote:
> On Tue, 21 Aug 2001 10:02:03, "Lavrentios Servissoglou"

> > Hello Peter,


> > (include some samples and a more specific description (Operating System,
> > version of REXX / Object REXX, ...)).

> > We try as soon as possible to understand and solve the problem.

> Can other ORexx problems be reported to this e-mail too?
> There is very {*filter*} semaphore leak in every version of OS/2 OREXX.
> Every RexxStart creates a number of semaphores. They are freed when
> program exits, but process that calls RexxStart in circle (i.e. for
> hooks) will either fail or even make the whole system unstable
> (because there are no free semaphores).
> I've tried reporting this using general IBM process and got usual "Not
> known problem", "Pay to fix" answer.

> --
> Bye,
> Vit

> P.S. Use tivv at ukr.net address to reply.



Fri, 13 Feb 2004 20:15:51 GMT  
 
 [ 14 post ] 

 Relevant Pages 

1. Tcl getting slower and slower on HP-UX

2. slow, very slow

3. Loop with Write key.vi goes slower and slower

4. Slow file access - Slow program startup

5. Form is slow getting up to speed

6. GETTING TO C5 EMBEDS IS VERY SLOW

7. canvas with >5000 items gets slow?

8. The Smalltalk Store: Why we've been slow, and why we're getting better

9. All the talk about slow servers, Is Novell slow too?

10. Pong from Inside Smalltalk is SLOW under Squeak

11. PositionableStream>>nextWord slow

12. My application is runing very slow...

 

 
Powered by phpBB® Forum Software