how to do 150,000 record associative arrays ??? 
Author Message
 how to do 150,000 record associative arrays ???


> I'm trying to do some perl processing of an employee database consisting of
> 150,000 people, each with colon-delimited one-line-per-person records.

> The problem I'm having is that I think I'm hitting a memory leak wall
> in having so many records in a huge set of associative arrays.  When
> I get to about 100,000 records, it hangs on an Apollo (HP/Apollo 64MB RAM,
> 400MB swap) and it also hangs (and core dumps) on a DEC Alpha (osf/1 and way
> more memory and swap).

It's not a leak in the strict sense. Perl's memory allocation
mechanisms do not cooperate well with hashes as they grow.

I would recommend you look at the DB_File module (or other DBM modules).

        perldoc AnyDBM_File

for information (with Perl5.001m).


Tue, 14 Apr 1998 03:00:00 GMT  
 [ 1 post ] 

 Relevant Pages 

1. Nested Associative Arrays, Can they be done?

2. Sorting associative array by record field (perl V4.036)

3. associative array or associative arrays

4. Associative array of associative arrays

5. Newbie : how do I create an array of records/arrays (Perl5)

6. (Q) array of associative arrays

7. Associative array of associate arrays

8. Array of Associative Arrays?

9. Q: using arrays in an associative array

10. Array of associative arrays?

11. associative arrays of arrays

12. Help with sort routine and array of associative arrays


Powered by phpBB® Forum Software