How to build an array of 700,000 records? 
Author Message
 How to build an array of 700,000 records?

Hi all,

Some time ago I sent a question to group regarding performance when drawing
a roadmap from a queue of roadsegment data. One of the surgestions was to
use an array instead of a queue to hold the data. Well my problem is that I
cannot build an array with 700,000 records.

Any ideas how to do this?

Regards,

Henrik Kartin
Kartin Data
Denmark




Sun, 02 Oct 2005 20:12:20 GMT  
 How to build an array of 700,000 records?
Use a queue?

Al

Quote:

> Hi all,

> Some time ago I sent a question to group regarding performance when drawing
> a roadmap from a queue of roadsegment data. One of the surgestions was to
> use an array instead of a queue to hold the data. Well my problem is that I
> cannot build an array with 700,000 records.

> Any ideas how to do this?

> Regards,

> Henrik Kartin
> Kartin Data
> Denmark





Sun, 02 Oct 2005 21:19:31 GMT  
 How to build an array of 700,000 records?
The problem with a queue is that it apparently produces a lot of overhead
when I have so many records. So to up performance I would like to try the
array idea because everyone tells me that it would be faster.

Henrik



Quote:
> Use a queue?

> Al


> > Hi all,

> > Some time ago I sent a question to group regarding performance when
drawing
> > a roadmap from a queue of roadsegment data. One of the surgestions was
to
> > use an array instead of a queue to hold the data. Well my problem is
that I
> > cannot build an array with 700,000 records.

> > Any ideas how to do this?

> > Regards,

> > Henrik Kartin
> > Kartin Data
> > Denmark





Sun, 02 Oct 2005 21:23:32 GMT  
 How to build an array of 700,000 records?


Quote:
> The problem with a queue is that it apparently produces a lot of overhead
> when I have so many records. So to up performance I would like to try the
> array idea because everyone tells me that it would be faster.

> Henrik



> > Use a queue?

> > Al


> > > Hi all,

> > > Some time ago I sent a question to group regarding performance when
> drawing
> > > a roadmap from a queue of roadsegment data. One of the surgestions was
> to
> > > use an array instead of a queue to hold the data. Well my problem is
> that I
> > > cannot build an array with 700,000 records.

> > > Any ideas how to do this?

> > > Regards,

> > > Henrik Kartin
> > > Kartin Data
> > > Denmark



In 32 bit app, array is unlimited.
Otherwise
    Group1    Group,Dim(100)
    Group2    Group,Dim(100)
    Group3    Group,Dim(100)
    Field  Any (or something else) !Field is Array(1000 000)
        End
        End
        End


Sun, 02 Oct 2005 21:51:42 GMT  
 How to build an array of 700,000 records?
Hi Henrik,

Quote:
> Some time ago I sent a question to group regarding performance when
drawing
> a roadmap from a queue of roadsegment data. One of the surgestions was to
> use an array instead of a queue to hold the data. Well my problem is that
I
> cannot build an array with 700,000 records.

What does your Queue structure look like, and how do you use it? Are you
sure that tweaking the record access code (going from Queue to Array) will
give you the speed increase you want. Is the Queue pre-sorted by the key
value you are looking on? Perhaps you are doing a lot of string moving
instead of passing strings by reference. I have seen dramatice speed
improvements by just letting strings sit in memory, and using a reference
for all my function calls. Perhaps you are doing alot of GUI drawing that
could be streamlined.

Looking to help,

Jim Gambon



Sun, 02 Oct 2005 23:37:38 GMT  
 How to build an array of 700,000 records?
Hi Henrik,

On Wed, 16 Apr 2003 14:12:20 +0200, "Henrik Kartin"

Quote:

>use an array instead of a queue to hold the data. Well my problem is that I
>cannot build an array with 700,000 records.

Why can't you build the array - what happens?  There should not be a
limit in 32bit applications to the array size (at least the docs say
there isn't<g>)

The following code works fine (1 million entries)

 Program
 Map
 End

A  Byte,Dim(1000000)
I  Long
S  Long
E  Long
 Code
 S  = Clock()
 Loop I = 1 To 1000000
   A[I] = 1
 End
 E = Clock()
 Message('It took ' & E-S & '/100 seconds to fill the array')

So does this one:

 Program
 Map
 End

A  Long,Dim(1000000)
I  Long
S  Long
E  Long
 Code
 S  = Clock()
 Loop I = 1 To 1000000
   A[I] = I
 End
 E = Clock()
 Message('It took ' & E-S & '/100 seconds to fill the array')

Takes a bit longer (about 5/100 seconds on my machine, where the byte
takes about 1/100 seconds)

Tried with a queue too:

 Program
 Map
 End

A  Long,Dim(1000000)
I  Long
S  Long
SS Long
E  Long
EE Long
Q  Queue
B    Long
   End
 Code
 S  = Clock()
 Loop I = 1 To 1000000
   A[I] = I
 End
 E = Clock()
 Message('It took ' & E-S & '/100 seconds to fill the array')

 SS = Clock()
 Loop I = 1 To 1000000
   Q.B = I
   Add(Q)
 End
 E = Clock()
 Free(Q)
 EE = Clock()
 Message('It took ' & E-SS & '/100 seconds to fill the queue and ' & EE-E & '/100 seconds to free it')

The array took about 3-5/100 seconds, filling the queue took 88/100
seconds and freeing it took 39/100 seconds, total 127/100.  So the
array is indeed considerably faster than the array in this very, very
simple testing.

Best regards,

Arnr Baldvinsson
Icetips Software        
San Antonio, Texas, USA
www.icetips.com

Subscribe to information from Icetips.com:
http://www.icetips.com/getnotificationinfo.htm



Mon, 03 Oct 2005 00:13:53 GMT  
 
 [ 6 post ] 

 Relevant Pages 

1. 700,000 File Transfers from the ASR

2. An array of records within an array of records (Still a problem)

3. an array of records within an array of records

4. convert 2d array to 1d array without using shift registers and build array

5. Arrays: Build array in multiple for loops or replace array elements

6. Building a "record" in Eiffel

7. Building a "record" in Eiffel

8. building binay record in TCL

9. Array Record Variables

10. Record Array

11. Adding records from an array

12. Get record in an array

 

 
Powered by phpBB® Forum Software