Help Help Speed Up Project 
Author Message
 Help Help Speed Up Project

I urgently need the help of an experienced  C programmer for several weeks!  

Currently I have a large CPD 2.110 system.  One routine which is
quite small computes intolerably slow.   I need this converted to C
with the intent of speeding it up considerably.  

Details

The existing CPD 2.110  routine reads a file consisting of about
150,000 records and processes it against a file of about 1/3 the
size.  Approximately  2500 pages of report are printed.  The
processing is not a simple serial pass of the files.  Key files are
used with the most heavily used file set up as a ramdisk file.  The program
works fine now but takes  over 30 hours to complete on a Pentium 200
 under DOS.  A 8 meg cache is used.  An ultra wide scsi controller
and mirrored drive is used.  Various other controllers and drives have been
 swaped in; all produce similar disappointing results.  

Analysis of bottlenecks show that the cache works well with a hit
rate of 96-98%.   Its clear that cpu power is the problem.   My
intuition says that the cpu usage actually consists of  math plus the
computation associated with all the keyfile  I/O.

Anyone experienced with integrating CPD2.110 and C programs ( or even
assembly), and who believe that the process can be speed up,  are invited to
respond to this well paying short term project.

Roger Clark



Fri, 01 Oct 1999 03:00:00 GMT  
 Help Help Speed Up Project

Quote:

> I urgently need the help of an experienced  C programmer for several weeks!

> Currently I have a large CPD 2.110 system.  One routine which is
> quite small computes intolerably slow.   I need this converted to C
> with the intent of speeding it up considerably.

> Details

> The existing CPD 2.110  routine reads a file consisting of about
> 150,000 records and processes it against a file of about 1/3 the
> size.  Approximately  2500 pages of report are printed.  The
> processing is not a simple serial pass of the files.  Key files are
> used with the most heavily used file set up as a ramdisk file.  The program
> works fine now but takes  over 30 hours to complete on a Pentium 200
>  under DOS.  A 8 meg cache is used.  An ultra wide scsi controller
> and mirrored drive is used.  Various other controllers and drives have been
>  swaped in; all produce similar disappointing results.

> Analysis of bottlenecks show that the cache works well with a hit
> rate of 96-98%.   Its clear that cpu power is the problem.   My
> intuition says that the cpu usage actually consists of  math plus the
> computation associated with all the keyfile  I/O.

> Anyone experienced with integrating CPD2.110 and C programs ( or even
> assembly), and who believe that the process can be speed up,  are invited to
> respond to this well paying short term project.

> Roger Clark


Are you using overlays? They really slow things down in programs such as
yours. One trick to speed things up is to run the program from the
ramdisk. I find the fastest result come from using no overlays and a
protected mode linker.(CAUSEWAY)

Goodluck
Gregg La Baw



Fri, 01 Oct 1999 03:00:00 GMT  
 Help Help Speed Up Project

Roger:
I hate to even suggest this because I don't want to offend anyone, but are
you using stream with those files? I have a rather complicated conversion
program that I use the process template to run the conversion. I was just
about ready to break down and hand code the whole thing, adding records and
then building the indexes at the end. Stream made one hell of a difference.
The conversion runs easily 3 times as fast, maybe more.

Mick



Quote:

> I urgently need the help of an experienced  C programmer for several
weeks!  

> Currently I have a large CPD 2.110 system.  One routine which is
> quite small computes intolerably slow.   I need this converted to C
> with the intent of speeding it up considerably.  

> Details

> The existing CPD 2.110  routine reads a file consisting of about
> 150,000 records and processes it against a file of about 1/3 the
> size.  Approximately  2500 pages of report are printed.  The
> processing is not a simple serial pass of the files.  Key files are
> used with the most heavily used file set up as a ramdisk file.  The
program
> works fine now but takes  over 30 hours to complete on a Pentium 200
>  under DOS.  A 8 meg cache is used.  An ultra wide scsi controller



Fri, 01 Oct 1999 03:00:00 GMT  
 Help Help Speed Up Project

Roger,
        Might be out of the question, but do you have access to CFD 3? You
might try to port it to that or one of the CW versions first. Maybe run
it as a stand alone EXE and at least see if it becomes reasonable. There
should be a marked improvement. I believe CFD 3 makes some fast EXE's (I
know that CW does!). If you don't have access to these products, you
might ask someone on line to make a quick EXE for you (you could always
do a RUN() from your base program). Just a thought...

        Happy hunting...

        kurtt



Fri, 01 Oct 1999 03:00:00 GMT  
 Help Help Speed Up Project

Roger

Quote:

> I urgently need the help of an experienced  C programmer for several
weeks!  

> Currently I have a large CPD 2.110 system.  One routine which is
> quite small computes intolerably slow.   I need this converted to C
> with the intent of speeding it up considerably.  

One thing you could do is go to version 3 which gives you a native exe.
Remember that when generating a report your printer is ALWAYS the
bottleneck. Faster processing will not improve your print speed. Maybe you
should look at your printer before you spend a lot of money on something
which may not help. Other than that it often pays to look for the obvious
eg. don't do any pocessing, computation or setting of variables within the
loop that can be done outside of it etc. Another worthwhile option may be
to generate a temporary file first and then print it (overnight)

HTH
Sean Gates
Gates Computer Systems
http://www.tradepage.co.za/gcs



Fri, 01 Oct 1999 03:00:00 GMT  
 Help Help Speed Up Project

Quote:
> Anyone experienced with integrating CPD2.110 and C programs ( or even
> assembly), and who believe that the process can be speed up,  are invited to
> respond to this well paying short term project.

Why don't you convert it to CFD 3.102? I'm sure you would get the
maximum possible performance and would still be able to maintain your
code..... You would also be able to change databases fairly easily to
find a better one maybe?

HTH,

Simon Brewer, Email Limited (Australia)



Fri, 01 Oct 1999 03:00:00 GMT  
 Help Help Speed Up Project

Quote:
>Currently I have a large CPD 2.110 system.  One routine which is
>quite small computes intolerably slow.   I need this converted to C
>with the intent of speeding it up considerably.  

What type of files are you reading?  DOS, ASCII, BASIC or Clarion DAT
files.  If Clarion Dat files C might not be a simple task.  I am not
sure if the dat file structs. and Key structs. are published or not.
If not you would need to write routines for all the basic file access.
The simple solution would be to use Clarion 3.1 dos.  With the Dos
Extender.  I used it as my tool of choice for data conversion programs
taking Banks mainframe files and converting them into other formats.
I think I had it doing about 50,000 records in just a few minutes.  

Quote:
>Details
>The existing CPD 2.110  routine reads a file consisting of about
>150,000 records and processes it against a file of about 1/3 the
>size.  Approximately  2500 pages of report are printed.

I dont know what type of printer you have but at 8 pages per min.  It
toulw take about 5 1/2 hours to print 2500 pages.  Does the printing
occur during the file reads?  Or it it done after all the data is
calculated?  Also where is it being spooled?  On the local printer or
over a network?

Quote:
> The
>processing is not a simple serial pass of the files.  Key files are
>used with the most heavily used file set up as a ramdisk file.  The program
>works fine now but takes  over 30 hours to complete on a Pentium 200
> under DOS.  A 8 meg cache is used.  An ultra wide scsi controller
>and mirrored drive is used.  Various other controllers and drives have been
> swaped in; all produce similar disappointing results.  

Is there a better way to structure the files?  Could you keep
subtotals in a temp file during data entry so that when you need to
run the reports you would not need to "recalculate" everything by
looking at 150,000 records * 50,000 records?

Quote:
>Analysis of bottlenecks show that the cache works well with a hit
>rate of 96-98%.   Its clear that cpu power is the problem.   My
>intuition says that the cpu usage actually consists of  math plus the
>computation associated with all the keyfile  I/O.
>Anyone experienced with integrating CPD2.110 and C programs ( or even
>assembly), and who believe that the process can be speed up,  are invited to
>respond to this well paying short term project.

Clarion 3.1 with Dos extender is reall fast.  But if it does not work
you may need to look how efficent your data structures are for this
report.  You might want look at how many "reads" it takes to get your
calculations.  There may be some ways to do some temp file
preprocessing to cut down on the number of "reads" the report needs to
do.

Just some ideas.

Troy
-----------------------------
Councelor Troy [TTS net]



Sat, 02 Oct 1999 03:00:00 GMT  
 Help Help Speed Up Project



Quote:

> I urgently need the help of an experienced  C programmer for several
weeks!  

> Currently I have a large CPD 2.110 system.  One routine which is
> quite small computes intolerably slow.   I need this converted to C
> with the intent of speeding it up considerably.  

So, you think C would do this any faster?

--
Tom Ruby
--------------------------------------------------------------------

http://www.netins.net/showcase/tomruby/
(It must be time to figure out a new tag line)



Sat, 02 Oct 1999 03:00:00 GMT  
 Help Help Speed Up Project

Troy,

I have similar times in my conversion programs (and I have a few of them
were APPEND is not allowed and I MUST use ADD.  Still very nice execution!)

--
Russell B. Eggen
Topspeed_Corporation.ConsultingDivision()

[Using MS Internet News on fortress.uccb.ns.ca]


Quote:
> The simple solution would be to use Clarion 3.1 dos.  With the Dos
> Extender.  I used it as my tool of choice for data conversion programs
> taking Banks mainframe files and converting them into other formats.
> I think I had it doing about 50,000 records in just a few minutes.  



Sat, 02 Oct 1999 03:00:00 GMT  
 
 [ 9 post ] 

 Relevant Pages 

1. Help Help Speed Up Project -Reply

2. UPS Maxicode syntax , HELP...

3. Help with debugger UPS

4. no help on pop-ups yet

5. fun project/ help me verify my graphics project!

6. help! f90.help help help help

7. ***HELP***HELP***NEED INFORMATION***HELP***HELP

8. HELP HELP HELP HELP

9. HELP HELP HELP HELP

10. Ord Function HELP Please HELP HELP HELP

11. help help help help!!!!!!!!!!!

12. (HELP (HELP (HELP (HELP))))

 

 
Powered by phpBB® Forum Software