Reporting performance 
Author Message
 Reporting performance

Hi all,

Shot in the dark here, but if there's an answer life would be much better...

I'm running an app in VFP7 that produces about 40,000 pages of output to pdf
and turns right around and does the same to postscript.  The problem I'm
running into is the act of producing the output itself accounts for about
90% of the total time the four hour program takes to run.  I've done some
testing and found some things like not swapping printer drivers back and
forth (Acrobat to PS to Acrobat to PS...) save some time, but still amounts
more to shaving time than cutting it.

Are there other areas to look at?  Perhaps more efficient drivers?  I've
tried using some API code to hide the Printing... window display but that
seemed more to mask what was going on than save on performance.

Here's hoping!

- John



Sun, 28 Aug 2005 07:09:04 GMT  
 Reporting performance
Hi John,
  It is difficult to suggest optimization techniques without knowing your
code.  However, as George Goley once said, to optimize, all you have to do
is take the slow parts out.
  Here is another suggestion that may cut your time in half: run the pdf and
postscript jobs on two separate machines running at the same time.
  Good luck!
  Regards, Chaim Caron (New York City)


Sun, 28 Aug 2005 09:17:16 GMT  
 Reporting performance
John,

You are already rendering at 5.5 pages per second (2 * 40000 / 4 / 3600). Is
each page written as a separate file? I'm not sure you are going to get much
faster. Make sure you have SET TALK OFF, aren't using WAIT windows etc. Are
you issuing 80k REPORT FORM commands or  just 2?

You can look for faster PDF and PS drivers and a faster box to run it on.

--
df - Microsoft MVP FoxPro http://www.geocities.com/df_foxpro


Quote:
> I'm running an app in VFP7 that produces about 40,000 pages of output to
pdf
> and turns right around and does the same to postscript.  The problem I'm
> running into is the act of producing the output itself accounts for about
> 90% of the total time the four hour program takes to run.  I've done some
> testing and found some things like not swapping printer drivers back and
> forth (Acrobat to PS to Acrobat to PS...) save some time, but still
amounts
> more to shaving time than cutting it.

> Are there other areas to look at?  Perhaps more efficient drivers?  I've
> tried using some API code to hide the Printing... window display but that
> seemed more to mask what was going on than save on performance.



Sun, 28 Aug 2005 10:31:54 GMT  
 Reporting performance

Quote:

>I'm running an app in VFP7 that produces about 40,000 pages of output to pdf
>and turns right around and does the same to postscript.  The problem I'm
>running into is the act of producing the output itself accounts for about
>90% of the total time the four hour program takes to run.  I've done some
>testing and found some things like not swapping printer drivers back and
>forth (Acrobat to PS to Acrobat to PS...) save some time, but still amounts
>more to shaving time than cutting it.

It occurs to me to ask where they hurry is.

Does the 40,000 pages comprise a single pdf file or 40,000 pdf files?

In the first 30 minutes, you are going to produce 5,000 pages; who is going to
need to read page 5,001 before you product the next 5,000 pages? So, for
example, one solution might be to produce subsets of the reports, initially, and
then the full reports afterwards. That might prompt a lateral solution to your
problem.

Are you able to provide some background about how  or what the reports are used
for?
regards

anthony shipley
-
Be a lert! Your country needs lerts.



Sun, 28 Aug 2005 13:53:09 GMT  
 Reporting performance
Have you considered printing just the postscript and the converting it to
PDF using a tool like ghostscript http://www.cs.wisc.edu/~ghost/ ?

HTH,
Remus


Quote:
> Hi all,

> Shot in the dark here, but if there's an answer life would be much
better...

> I'm running an app in VFP7 that produces about 40,000 pages of output to
pdf
> and turns right around and does the same to postscript.  The problem I'm
> running into is the act of producing the output itself accounts for about
> 90% of the total time the four hour program takes to run.  I've done some
> testing and found some things like not swapping printer drivers back and
> forth (Acrobat to PS to Acrobat to PS...) save some time, but still
amounts
> more to shaving time than cutting it.

> Are there other areas to look at?  Perhaps more efficient drivers?  I've
> tried using some API code to hide the Printing... window display but that
> seemed more to mask what was going on than save on performance.

> Here's hoping!

> - John



Sun, 28 Aug 2005 14:24:43 GMT  
 Reporting performance
Thanks, all for getting involved!  I thought it easier to respond in general
for the most part.

A bit more background:  The app is a billing system used in-house in one
sweeping run each month.  We keep a pdf copy of every bill for customer
service purposes as well as providing about a third of our customers with an
ebill.  The rest get dumped off into a 1 GB postscript file that is sent to
a printshop.

The process itself is essentially spinning through a table of current
customers.  For each customer, at least two and as many as about 15
different page layouts may be used.  So the 40,000 pages are coming from
somewhere on the order of 20,000 REPORT FORMs.  The postscript is
concatenated onto the end of a text file right after creation.  In a later
step, the roughly 10,000 PDFs are merged into about 4,000 files: one per
customer.

As far as the code, most of it pulls together totals for a customer's
account, fires summary reports, then gathers any supporting details and
produces line items and departmental summary type reports.  On average,
there is probably less than one UDF per report, though a few are called
repeatedly in detail bands.  For the most part, it still seems more the
sending through printer drivers that takes the most time.

David mentioned:

Quote:
>You can look for faster PDF and PS drivers and a faster box to run it on.

That was one of the issues prompting me to post originally.  Anyone know of
a good resource for such drivers?

Chaim:  I've been working on a similar solution to your suggestion of
splitting the work out.  It has a lot of promise.  The minor overhead of
writing the results out to a more permanent and reconstructable format
should easily be a worthwhile tradeoff.

Remus, the postscript conversion could very possibly be of use.  Thanks for
the suggestion!

Anthony posed:

Quote:
> It occurs to me to ask where they hurry is.

For various reasons, some legacy, some reality, some just not yet having had
time to do it better, we don't have final data prepared for running the
bills until at least the third business day of the month, usually later.
Shortly pusuant to that time the President and CEO (two separate personages)
start coming around getting agitated about bills not going out yet.  (As an
aside: these are the same two who unleashed a new business unit requiring
about 500 hours of retrofitting on the billing system--three weeks before it
had to be in production.  Not too bad if your programming staff had more
than one person!)  So I'm sitting out here on the wing of the plane, in
flight, trying to redesign the engine.

Whining aside, saving two hours gives us a two hour head start on continuing
the process--one in four times this could mean the difference between the
bills getting dropped in the mail today vs. tomorrow.  The bigger issue is
that we find ourselves having to rerun the process, after doing final checks
and discovering some error.  With the process down to around two hours we
could conceivably make three attempts in a day, four leaves us as at best
with two in most cases.

Also, while doing as Chaim mentioned (producing data on one box while
printing on another) lateralizes to some degree, the printshop needs the
entire dataset before it can really start doing its stuff, particularly for
postal sorting considerations.


Quote:
> Hi all,

> Shot in the dark here, but if there's an answer life would be much
better...

> I'm running an app in VFP7 that produces about 40,000 pages of output to
pdf
> and turns right around and does the same to postscript.  The problem I'm
> running into is the act of producing the output itself accounts for about
> 90% of the total time the four hour program takes to run.  I've done some
> testing and found some things like not swapping printer drivers back and
> forth (Acrobat to PS to Acrobat to PS...) save some time, but still
amounts
> more to shaving time than cutting it.

> Are there other areas to look at?  Perhaps more efficient drivers?  I've
> tried using some API code to hide the Printing... window display but that
> seemed more to mask what was going on than save on performance.

> Here's hoping!

> - John



Mon, 29 Aug 2005 00:42:58 GMT  
 
 [ 6 post ] 

 Relevant Pages 

1. VFP performance concerns/proof

2. Indexes and performance

3. VFP Performance sluggish?!

4. Performance Issue when opening files

5. Thread on Rushmore performance?

6. ODBC performance, SQLSetPos, and MSKB #Q126131

7. ODBC Performance Problem

8. How to enhance the performance?

9. Form loading performance

10. Form/Pageframe/Grid Performance

11. Performance Utilities for Fox (DOS)

12. A Performance Question

 

 
Powered by phpBB® Forum Software