perl -pe "s/<er>/<er>\n"/g": memory efficient version? 
Author Message
 perl -pe "s/<er>/<er>\n"/g": memory efficient version?

I have a file which has 1 super long line that contains

some thing<er>something else<er>and more stuff<er>...

perl -pe "s/<er>/<er>\n"/g"

works for small files, but for large files, it runs out of
memory. Is there a memory efficient version?

awk '{gsub("<er>","<er>\n"); print}'
is not memory efficient either.

Any solution using standard Unix utility is welcome,
(without having to code a control character).

Thanks.
--



Wed, 26 Oct 2005 06:09:51 GMT  
 perl -pe "s/<er>/<er>\n"/g": memory efficient version?

: I have a file which has 1 super long line that contains
:
: some thing<er>something else<er>and more stuff<er>...
:
: perl -pe "s/<er>/<er>\n"/g"
:
: works for small files, but for large files, it runs out of
: memory. Is there a memory efficient version?

    perl -lpe "BEGIN{$/='<er>'}" foo.txt



Wed, 26 Oct 2005 06:18:01 GMT  
 perl -pe "s/<er>/<er>\n"/g": memory efficient version?


:
: : I have a file which has 1 super long line that contains
: :
: : some thing<er>something else<er>and more stuff<er>...
: :
: : perl -pe "s/<er>/<er>\n"/g"
: :
: : works for small files, but for large files, it runs out of
: : memory. Is there a memory efficient version?
:
:     perl -lpe "BEGIN{$/='<er>'}" foo.txt

I should have tested that first.  I forgot about the auto-chomp()
behavior with that combination of switches.

    perl -pe "BEGIN{$\=$/;$/='<er>'}" foo.txt



Wed, 26 Oct 2005 06:24:20 GMT  
 perl -pe "s/<er>/<er>\n"/g": memory efficient version?



Quote:
> I have a file which has 1 super long line that contains

> some thing<er>something else<er>and more stuff<er>...

> perl -pe "s/<er>/<er>\n"/g"

> works for small files, but for large files, it runs out of
> memory.

> Any solution using standard Unix utility is welcome.

use tr to replace any > by a newline, then post-process with sed.

Laurent



Wed, 26 Oct 2005 09:18:14 GMT  
 perl -pe "s/<er>/<er>\n"/g": memory efficient version?

Quote:

> I have a file which has 1 super long line that contains

> some thing<er>something else<er>and more stuff<er>...

> perl -pe "s/<er>/<er>\n"/g"

> works for small files, but for large files, it runs out of
> memory. Is there a memory efficient version?

> awk '{gsub("<er>","<er>\n"); print}'
> is not memory efficient either.

> Any solution using standard Unix utility is welcome,
> (without having to code a control character).

perl -lpe'BEGIN{ $\=($/="<er>").$\ }' yourfile

John
--
use Perl;
program
fulfillment



Wed, 26 Oct 2005 13:20:37 GMT  
 perl -pe "s/<er>/<er>\n"/g": memory efficient version?

Quote:



>> I have a file which has 1 super long line that contains

>> some thing<er>something else<er>and more stuff<er>...

>> perl -pe "s/<er>/<er>\n"/g"

>> works for small files, but for large files, it runs out of
>> memory.

>> Any solution using standard Unix utility is welcome.

>use tr to replace any > by a newline, then post-process with sed.

I have ">" in between <er>'s.
--



Wed, 26 Oct 2005 20:24:11 GMT  
 perl -pe "s/<er>/<er>\n"/g": memory efficient version?

Quote:

> I should have tested that first.  I forgot about the
> auto-chomp() behavior with that combination of switches.

>     perl -pe "BEGIN{$\=$/;$/='<er>'}" foo.txt

I'm not sure you tested it this time either.  It converts

    --------
    aa<er>bb
    --------

into

    --------
    aa<er>
    bb

    --------

which has one newline too many.  Here are some that work:

    perl -pe 'BEGIN{ $/ = "<er>" } $_ .= "\n" if /<er>/
    perl -pe 'BEGIN{ $/ = "<er>" } s/<er>/<er>\n/' infile
    perl -pe 'BEGIN{ $/ = "<er>" } $_ .= "\n" if substr($_, -4) eq $/'

Peter

--
I wish dialog boxes had a button saying "Whatever".  I hate being
forced to answer "Yes" or "No" to a question I have no opinion on
whatsoever.  There ought to be a button matching my indifference.



Thu, 27 Oct 2005 12:38:28 GMT  
 perl -pe "s/<er>/<er>\n"/g": memory efficient version?

Quote:
> perl -lpe'BEGIN{ $\=($/="<er>").$\ }' yourfile

That fails if the last field does not end with "<er>".  The file

    --------
    aa<er>bb
    --------

becomes

    --------
    aa<er>
    bb
    <er>
    --------

which has one "<er>" too many.  The simplest solution is just a
slight modification of the origial version:

    perl -pe 'BEGIN{ $/ = "<er>" } s/<er>/<er>\n/'

Peter

--
I wish dialog boxes had a button saying "Whatever".  I hate being
forced to answer "Yes" or "No" to a question I have no opinion on
whatsoever.  There ought to be a button matching my indifference.



Thu, 27 Oct 2005 12:42:40 GMT  
 
 [ 8 post ] 

 Relevant Pages 

1. <<<<<< HELP >>>>>>>>

2. One question about useage "<>"

3. My <!--#exec cgi="\scripts\script.pl"-->command will not work

4. system("ed <stuff>")

5. About <SCRIPT language="PerlScript">

6. Text() ignored bind("<Tab>")

7. print <<"EOF";

8. print <<"ending_print_tag";

9. <<ENFofFTP "here-documnet" question

10. HELP print "<?xml version="1.0"?>"

11. Perl equivalent to UNIX shell ". <path><script>.sh"

 

 
Powered by phpBB® Forum Software