memory efficiency question 
Author Message
 memory efficiency question

When reading from a filehandle, does

while <FILEHANDLE>

 read the whole file into MEMORY at once or one line at a time?  Just
wondering if this is more memory efficient than;

David



Mon, 09 Oct 2000 03:00:00 GMT  
 memory efficiency question

Quote:

> When reading from a filehandle, does
> while <FILEHANDLE>
>  read the whole file into MEMORY at once or one line at a time?

One line at a time.

Quote:
> Just wondering if this is more memory efficient than;


It is. A lot.
--
                    Calle Dybedahl, UNIX Sysadmin



Mon, 09 Oct 2000 03:00:00 GMT  
 memory efficiency question

Quote:


> > When reading from a filehandle, does
> > while <FILEHANDLE>
> >  read the whole file into MEMORY at once or one line at a time?

> One line at a time.

> > Just wondering if this is more memory efficient than;

> It is. A lot.

It's also worth pointing out that
        while (<F>)
is more memory-efficient that
        for (<F>)
since the latter is essentially equivalent to


(except the temporary array is anonymous.)

John Porter



Mon, 09 Oct 2000 03:00:00 GMT  
 memory efficiency question

Quote:

> When reading from a filehandle, does

> while <FILEHANDLE>

>  read the whole file into MEMORY at once or one line at a time?  Just

This reads your file one line at a time. The end of a line is defined by
the $\ variable.

Quote:
> wondering if this is more memory efficient than;


> David

 This loads the entire file into an array. This should be faster, but if
your file is large, you might have memory problems ...
Using either approaches really depends on your application.

Hope that was helpful,

--
Ala Qumsieh             |  No .. not just another
ASIC Design Engineer    |  Perl Hacker!!!!!
Matrox Graphics Inc.    |
Montreal, Quebec        |  (Not yet!)



Fri, 27 Oct 2000 03:00:00 GMT  
 memory efficiency question


: >
: > David

:  This loads the entire file into an array. This should be faster, but if
: your file is large, you might have memory problems ...
: Using either approaches really depends on your application.

You'd better benchmark that! I assumed the same thing and found that line
at a time was somewhat faster than loading into an array! Reading the whole
file into a scalar, however, was very fast...

-- Tom Mornini
-- InfoMania



Sat, 28 Oct 2000 03:00:00 GMT  
 memory efficiency question



Quote:
> This reads your file one line at a time. The end of a line is defined by
> the $\ variable.

Do you mean the $/ variable?

# perldoc perlvar
[snip]
     $\      The output record separator for the print operator.
[snip]
     $/      The input record separator, newline by default.
[snip]

Martien
--
Martien Verbruggen                  |
Webmaster www.tradingpost.com.au    | If it isn't broken, it doesn't have
Commercial Dynamics Pty. Ltd.       | enough features yet.
NSW, Australia                      |



Sat, 28 Oct 2000 03:00:00 GMT  
 memory efficiency question



|

| : >
| : > David
|
| :  This loads the entire file into an array. This should be faster, but if
| : your file is large, you might have memory problems ...
| : Using either approaches really depends on your application.
|
| You'd better benchmark that! I assumed the same thing and found that line
| at a time was somewhat faster than loading into an array! Reading the whole
| file into a scalar, however, was very fast...

So what is the fastest way? What code did you use to read the file
directly to scalar?

    1.

    2.

        while ( <FILE> )
        {

        }

    3.

        $file = join '', <FILE>;

jari



Mon, 30 Oct 2000 03:00:00 GMT  
 memory efficiency question

Quote:
> | You'd better benchmark that! I assumed the same thing and found that line
> | at a time was somewhat faster than loading into an array! Reading the whole
> | file into a scalar, however, was very fast...

> So what is the fastest way? What code did you use to read the file
> directly to scalar?

{
        local $/ = undef;
        $file = <FILE>;
Quote:
}



Mon, 30 Oct 2000 03:00:00 GMT  
 memory efficiency question


|
| > | You'd better benchmark that! I assumed the same thing and found that line
| > | at a time was somewhat faster than loading into an array!
| > |Reading the whole
| > | file into a scalar, however, was very fast...
|
| {
|       local $/ = undef;
|       $file = <FILE>;
| }

I can't see the boost? Is my benchmark somehow faulty? The test file was
roughly 100k. Perl 5.4_4 HP_UX 9000/712 10.20

jari
            use Benchmark;

            timethese( 10000,
            {
                a =>

                ,
                b =>
                '$/ = undef;

                ,
                c =>
                '$/ = undef;
                 open F, "test.txt"; $file = <F>; close F'
            });

            a: 16 secs ( 2.37 usr  5.37 sys =  7.74 cpu)
            b: 16 secs ( 2.77 usr  5.49 sys =  8.26 cpu)
            c: 16 secs ( 2.14 usr  5.39 sys =  7.53 cpu)



Fri, 03 Nov 2000 03:00:00 GMT  
 
 [ 9 post ] 

 Relevant Pages 

1. Another question about memory efficiency

2. perl memory efficiency

3. Efficiency question

4. Question about efficiency

5. Perl5 objects & efficiency questions

6. Perl efficiency question

7. some questions about array refs & efficiency

8. Quick Question on Efficiency

9. run-time eval efficiency question

10. Time-efficiency question

11. Bizarre efficiency question.

12. Rookie question on efficiency of split/join vs loop

 

 
Powered by phpBB® Forum Software