Optimising Browse Load with Buffering using STREAM and FLUSH 
Author Message
 Optimising Browse Load with Buffering using STREAM and FLUSH

In a recent news group item (Observation 28/04/03) there was some talk from
Andy at Plugware about how he optimised BROWSES some years back using STREAM
and FLUSH - this reduces the number of database reads by getting several
records at once.  The concept is a sound one and I would have thought for
those of us die-hards (or scared to use SQL) that this could potentially
increase performance significantly in a networking environment.  Andy did
kinda describe to me how it could be done (he has not done it with the
latest ABC stuff) - I don't have the time it takes right now to try it out -
so I was wondering if anybody else had tried this.

Andy's thought were something along the following lines (sorry Andy if I am
wrong!).

We need to find the code points/loop where the table is read - in the loop
we need to insert a STREAM call and then a subsequent FLUSH call.  Andy
suggested using a modulus function so that say every 50 records it FLUSHES
and STREAMS again.  We would need an initial STREAM before the loop and I
guess a final FLUSH at the end of the loop.

What thoughts or experience does anybody have?



Mon, 17 Oct 2005 19:51:59 GMT  
 Optimising Browse Load with Buffering using STREAM and FLUSH
Hi Gavin,

Quote:
> Andy at Plugware about how he optimised BROWSES some years back using
STREAM
> and FLUSH - this reduces the number of database reads by getting several
> records at once.

If you are using ABC you might want to use the ResetQueue method. Try
putting the STREAM(filelabel) statements before the parent call in
ResetQueue. Then put the FLUSH(filelabel) statements after the parent call.
I had some good luck with this in the past, but I cannot seem to find my
code right now to verify. Make sure you do not FLUSH the view, just the
files that make up the view.

If it is a page loaded browse I don't see the need to FLUSH every so often.
If it is a file loaded browse then you might want to do the partial flushes
in the browses NEXT and PREVIOUS methods. You can create a local variable to
the procedure for the current read count, or add a property to the browse
class.

Good luck with it. I hope it helps out.

Regards,

Jim Gambon



Mon, 17 Oct 2005 20:44:12 GMT  
 Optimising Browse Load with Buffering using STREAM and FLUSH
STREAM and FLUSH can dramatically increase the performance on TPS files
however the do also lock the file at the same time which is a bit of a no no
in a multi user environment.  I've always wondered why QUICKSCAN was not
supported for TPS files.

Cheers

--
Ian Holdsworth
Senior Programmer / Assistant IT Manager
Response Analysis & Mailing Ltd

---------------------------------------------------

Direct Line:       +44 (0) 20 8880 8866
Switch Board:      +44 (0) 20 8880 8140
Fax:               +44 (0) 0870 134 0987

Web Site:          www.ram-ltd.co.uk


Quote:
> In a recent news group item (Observation 28/04/03) there was some talk
from
> Andy at Plugware about how he optimised BROWSES some years back using
STREAM
> and FLUSH - this reduces the number of database reads by getting several
> records at once.  The concept is a sound one and I would have thought for
> those of us die-hards (or scared to use SQL) that this could potentially
> increase performance significantly in a networking environment.  Andy did
> kinda describe to me how it could be done (he has not done it with the
> latest ABC stuff) - I don't have the time it takes right now to try it
out -
> so I was wondering if anybody else had tried this.

> Andy's thought were something along the following lines (sorry Andy if I
am
> wrong!).

> We need to find the code points/loop where the table is read - in the loop
> we need to insert a STREAM call and then a subsequent FLUSH call.  Andy
> suggested using a modulus function so that say every 50 records it FLUSHES
> and STREAMS again.  We would need an initial STREAM before the loop and I
> guess a final FLUSH at the end of the loop.

> What thoughts or experience does anybody have?



Mon, 17 Oct 2005 20:23:05 GMT  
 Optimising Browse Load with Buffering using STREAM and FLUSH
In a multi-user enviroment you really should use LogOut and Commit. I
have never used these in loading a view but use them for processes and
also for writing records from a form.
James Neill
Quote:

> STREAM and FLUSH can dramatically increase the performance on TPS files
> however the do also lock the file at the same time which is a bit of a no no
> in a multi user environment.  I've always wondered why QUICKSCAN was not
> supported for TPS files.

> Cheers

> --
> Ian Holdsworth
> Senior Programmer / Assistant IT Manager
> Response Analysis & Mailing Ltd

> ---------------------------------------------------

> Direct Line:       +44 (0) 20 8880 8866
> Switch Board:      +44 (0) 20 8880 8140
> Fax:               +44 (0) 0870 134 0987

> Web Site:          www.ram-ltd.co.uk



> > In a recent news group item (Observation 28/04/03) there was some talk
>  from
> > Andy at Plugware about how he optimised BROWSES some years back using
>  STREAM
> > and FLUSH - this reduces the number of database reads by getting several
> > records at once.  The concept is a sound one and I would have thought for
> > those of us die-hards (or scared to use SQL) that this could potentially
> > increase performance significantly in a networking environment.  Andy did
> > kinda describe to me how it could be done (he has not done it with the
> > latest ABC stuff) - I don't have the time it takes right now to try it
>  out -
> > so I was wondering if anybody else had tried this.

> > Andy's thought were something along the following lines (sorry Andy if I
>  am
> > wrong!).

> > We need to find the code points/loop where the table is read - in the loop
> > we need to insert a STREAM call and then a subsequent FLUSH call.  Andy
> > suggested using a modulus function so that say every 50 records it FLUSHES
> > and STREAMS again.  We would need an initial STREAM before the loop and I
> > guess a final FLUSH at the end of the loop.

> > What thoughts or experience does anybody have?



Tue, 18 Oct 2005 01:03:14 GMT  
 Optimising Browse Load with Buffering using STREAM and FLUSH
Well -- IF SV would provide a FileManager Property to "Disable" Optimistic
file locking then this would solve a long reported problem...

For Browses and Reports (usually) No updates are done (or intended) so
"Dirty" Reads are fine.....

Kevin Erskine


Quote:
> In a multi-user enviroment you really should use LogOut and Commit. I
> have never used these in loading a view but use them for processes and
> also for writing records from a form.
> James Neill




Quote:
> > STREAM and FLUSH can dramatically increase the performance on TPS files
> > however the do also lock the file at the same time which is a bit of a
no no
> > in a multi user environment.  I've always wondered why QUICKSCAN was not
> > supported for TPS files.

> > Cheers

> > --
> > Ian Holdsworth
> > Senior Programmer / Assistant IT Manager
> > Response Analysis & Mailing Ltd

> > ---------------------------------------------------

> > Direct Line:       +44 (0) 20 8880 8866
> > Switch Board:      +44 (0) 20 8880 8140
> > Fax:               +44 (0) 0870 134 0987

> > Web Site:          www.ram-ltd.co.uk



> > > In a recent news group item (Observation 28/04/03) there was some talk
> >  from
> > > Andy at Plugware about how he optimised BROWSES some years back using
> >  STREAM
> > > and FLUSH - this reduces the number of database reads by getting
several
> > > records at once.  The concept is a sound one and I would have thought
for
> > > those of us die-hards (or scared to use SQL) that this could
potentially
> > > increase performance significantly in a networking environment.  Andy
did
> > > kinda describe to me how it could be done (he has not done it with the
> > > latest ABC stuff) - I don't have the time it takes right now to try it
> >  out -
> > > so I was wondering if anybody else had tried this.

> > > Andy's thought were something along the following lines (sorry Andy if
I
> >  am
> > > wrong!).

> > > We need to find the code points/loop where the table is read - in the
loop
> > > we need to insert a STREAM call and then a subsequent FLUSH call.
Andy
> > > suggested using a modulus function so that say every 50 records it
FLUSHES
> > > and STREAMS again.  We would need an initial STREAM before the loop
and I
> > > guess a final FLUSH at the end of the loop.

> > > What thoughts or experience does anybody have?



Tue, 18 Oct 2005 01:25:05 GMT  
 Optimising Browse Load with Buffering using STREAM and FLUSH

Quote:

> Well -- IF SV would provide a FileManager Property to "Disable" Optimistic
> file locking then this would solve a long reported problem...

Changing these settings requires a reboot of Win2k/XP. Not sure they have
the ability to do this in the driver.

If they do, itd be a nice option:)

--
Mark Riffey
Granite Bear
http://www.granitebear.com



Mon, 17 Oct 2005 19:53:18 GMT  
 Optimising Browse Load with Buffering using STREAM and FLUSH
Well why can't  they open the file in read-only - doesn't this accomplish
the same thing?


Quote:

> > Well -- IF SV would provide a FileManager Property to "Disable"
Optimistic
> > file locking then this would solve a long reported problem...

> Changing these settings requires a reboot of Win2k/XP. Not sure they have
> the ability to do this in the driver.

> If they do, itd be a nice option:)

> --
> Mark Riffey
> Granite Bear
> http://www.granitebear.com



Tue, 18 Oct 2005 02:08:43 GMT  
 Optimising Browse Load with Buffering using STREAM and FLUSH

Quote:

> Well why can't  they open the file in read-only - doesn't this accomplish
> the same thing?

It might. I dunno how the locks function. Ie: do they care if the file is
open read only? Surf might know.

--
Mark Riffey
Granite Bear
http://www.granitebear.com



Mon, 17 Oct 2005 21:23:20 GMT  
 Optimising Browse Load with Buffering using STREAM and FLUSH
Mark,et al
There is a small program available on our web site that checks, and
optionally sets the correct registry options (I haven't actually gone and
checked to verify it).  You can download from
ftp://www.keystonecr.com/demos/redrtest.exe  I found it on the net here:
http://www.softwaretechnology.biz/download.html
Regards
Randy


Quote:

> > Well -- IF SV would provide a FileManager Property to "Disable"
Optimistic
> > file locking then this would solve a long reported problem...

> Changing these settings requires a reboot of Win2k/XP. Not sure they have
> the ability to do this in the driver.

> If they do, itd be a nice option:)

> --
> Mark Riffey
> Granite Bear
> http://www.granitebear.com



Tue, 18 Oct 2005 11:17:29 GMT  
 Optimising Browse Load with Buffering using STREAM and FLUSH
I have one from bytemasters that does the same thing. Still requires a
reboot tho.

--
Mark Riffey
Granite Bear
http://www.granitebear.com



Tue, 18 Oct 2005 07:08:17 GMT  
 Optimising Browse Load with Buffering using STREAM and FLUSH
Thanks Jim, that sounds like the right places to start - I shall give it a
go.
Regards
Gavin


Quote:
> Hi Gavin,

> > Andy at Plugware about how he optimised BROWSES some years back using
> STREAM
> > and FLUSH - this reduces the number of database reads by getting several
> > records at once.

> If you are using ABC you might want to use the ResetQueue method. Try
> putting the STREAM(filelabel) statements before the parent call in
> ResetQueue. Then put the FLUSH(filelabel) statements after the parent
call.
> I had some good luck with this in the past, but I cannot seem to find my
> code right now to verify. Make sure you do not FLUSH the view, just the
> files that make up the view.

> If it is a page loaded browse I don't see the need to FLUSH every so
often.
> If it is a file loaded browse then you might want to do the partial
flushes
> in the browses NEXT and PREVIOUS methods. You can create a local variable
to
> the procedure for the current read count, or add a property to the browse
> class.

> Good luck with it. I hope it helps out.

> Regards,

> Jim Gambon



Tue, 18 Oct 2005 16:46:22 GMT  
 Optimising Browse Load with Buffering using STREAM and FLUSH
Gavin,

Please let us know how you get on.
--
Regards,

David Groves
Alchemy Technology


Quote:
> Thanks Jim, that sounds like the right places to start - I shall give it a
> go.
> Regards
> Gavin



> > Hi Gavin,

> > > Andy at Plugware about how he optimised BROWSES some years back using
> > STREAM
> > > and FLUSH - this reduces the number of database reads by getting
several
> > > records at once.

> > If you are using ABC you might want to use the ResetQueue method. Try
> > putting the STREAM(filelabel) statements before the parent call in
> > ResetQueue. Then put the FLUSH(filelabel) statements after the parent
> call.
> > I had some good luck with this in the past, but I cannot seem to find my
> > code right now to verify. Make sure you do not FLUSH the view, just the
> > files that make up the view.

> > If it is a page loaded browse I don't see the need to FLUSH every so
> often.
> > If it is a file loaded browse then you might want to do the partial
> flushes
> > in the browses NEXT and PREVIOUS methods. You can create a local
variable
> to
> > the procedure for the current read count, or add a property to the
browse
> > class.

> > Good luck with it. I hope it helps out.

> > Regards,

> > Jim Gambon



Sat, 22 Oct 2005 15:22:41 GMT  
 
 [ 12 post ] 

 Relevant Pages 

1. Flushing stream buffers ?

2. Flushing buffered streams

3. How to FLUSH buffer using FPUTC/FGETC

4. stream & flush

5. logout/commit Vs stream/flush

6. Stream/Flush

7. SQL- Stream(File)/Flush(File)

8. Stream / Flush faster?

9. Stream & Flush question

10. Btrieve driver - Stream() and Flush() ?

11. Need info on STREAM() and FLUSH()

12. Stream & Flush

 

 
Powered by phpBB® Forum Software