sed question: unreliable "/^$/d" ? 
Author Message
 sed question: unreliable "/^$/d" ?

Hello,
I hop this question would still interest the group, though it concerns
sed. I have a large (500+) group of text files that my unix system isn't
reading well, mostly errors in spaces, blank lines etc. The /^$/d
command doesn't seems to snag the blank lines -- has anyone else had
this problem?

Any suggestions appreciated -- thanks!

Drew Krause



Mon, 30 Jun 2003 16:33:46 GMT  
 sed question: unreliable "/^$/d" ?
        In Article 16933 of comp.lang.awk, Drew Krause

Quote:

> I hop this question would still interest the group, though it concerns
> sed. I have a large (500+) group of text files that my unix system isn't
> reading well, mostly errors in spaces, blank lines etc. The /^$/d
> command doesn't seems to snag the blank lines -- has anyone else had
> this problem?

        With /^$/d you're assuming the blank lines have nothing
in them when they probably have some sort of white space, i.e.
blank(s) or tab(s).  To make the pattern more complete to search
for such cases use:

        /^[     ]$/d

        where inside the [] are a space and tab

This will not only remove lines with nothing but also those
containing nothing but blanks and/or tabs.



Mon, 30 Jun 2003 15:59:42 GMT  
 sed question: unreliable "/^$/d" ?

Quote:

> This will not only remove lines with nothing but also those
> containing nothing but blanks and/or tabs.

Thanks for your reply. Yes, I tried your suggestion -- no luck. Even more
puzzling, these lines are truly blank (ie I can arrow-through them in an
editor and no spaces appear). Could this be a file-conversion problem; i.e.
not all text files are alike?=


Tue, 01 Jul 2003 01:27:52 GMT  
 sed question: unreliable "/^$/d" ?

Quote:


> > This will not only remove lines with nothing but also those
> > containing nothing but blanks and/or tabs.

> Thanks for your reply. Yes, I tried your suggestion -- no luck. Even more
> puzzling, these lines are truly blank (ie I can arrow-through them in an
> editor and no spaces appear). Could this be a file-conversion problem; i.e.
> not all text files are alike?=

Did any of these files come another computer system?
Primarily, are you on a *nix system and processing files
that may have come from MS operatiing systems?

If so, you may be having problems with the linefeed-carriage
return pair from msdos.  Look at a file like so:

        vi -b filename

If you see ^m at the end of every line, this is your problem.

Good luck,

John



Tue, 01 Jul 2003 01:04:38 GMT  
 sed question: unreliable "/^$/d" ?

Quote:

>    With /^$/d you're assuming the blank lines have nothing
>in them when they probably have some sort of white space, i.e.
>blank(s) or tab(s).  To make the pattern more complete to search
>for such cases use:

>    /^[     ]$/d

>    where inside the [] are a space and tab

>This will not only remove lines with nothing but also those
>containing nothing but blanks and/or tabs.

Whoops, there's a slight typo here that's causing weirdness.  The
correct sed command should read:

        /^[     ]*$/d

Note the ``*'', which says, ``match 0 or more of what came
immediately before me.''

I keep a script in my directory called ``noblanks'' that does
this; it's handy for, eg, removing blank lines from source code
and so on.

        - Dan C.



Tue, 01 Jul 2003 01:20:00 GMT  
 sed question: unreliable "/^$/d" ?

Quote:
> If so, you may be having problems with the linefeed-carriage
> return pair from msdos.  Look at a file like so:

>         vi -b filename

> If you see ^m at the end of every line, this is your problem.

The "^M"s *did* appear. How can I get rid of them?

Thanks for your help!



Tue, 01 Jul 2003 04:55:58 GMT  
 sed question: unreliable "/^$/d" ?


Quote:

>> If so, you may be having problems with the linefeed-carriage
>> return pair from msdos.  Look at a file like so:

>>         vi -b filename

>> If you see ^m at the end of every line, this is your problem.

>The "^M"s *did* appear. How can I get rid of them?

tr -d "\r" < inputfile > outputfile

will strip all carridge returns, not just at the end of 'blank' lines.
In one sed command and retaining carridge returns not immediately
preceding line feeds,

sed -e 's/\r$//' -e '/^[ \t]*$/d' inputfile > outputfile

Sent via Deja.com
http://www.deja.com/



Tue, 01 Jul 2003 04:40:30 GMT  
 sed question: unreliable "/^$/d" ?
Who ever said Drew Krause couldn't write what follows?:

Quote:
>Hello,
>I hop this question would still interest the group, though it concerns
>sed. I have a large (500+) group of text files that my unix system isn't
>reading well, mostly errors in spaces, blank lines etc. The /^$/d
>command doesn't seems to snag the blank lines -- has anyone else had
>this problem?

>Any suggestions appreciated -- thanks!

>Drew Krause

Make sure there are no tabs or spaces in the blank lines. I think the regexp
"/^[:space:]$/" ought to fix that. I'm not quite sure if sed handles the [::]
style regexp's.

--

Bryan Hoyt

http://www.crosswinds.net/~artmusic

-----------------------------------

Old musicians never die, they just decompose.



Wed, 09 Jul 2003 10:43:04 GMT  
 sed question: unreliable "/^$/d" ?


Quote:
>I hop this question would still interest the group, though it concerns
>sed. I have a large (500+) group of text files that my unix system
>isn't reading well, mostly errors in spaces, blank lines etc. The /^$/d
>command doesn't seems to snag the blank lines -- has anyone else had
>this problem?

Are you sure there aren't extra characters? If these files come from a
system running Windows then there's a high likelihood they have carridge
returns (ASCII 0x0D) just before each unix newline (ASCII 0x0A).

Try  sed '/^[ \t]*\r?$/d'  to remove lines that appear blank.

Sent via Deja.com
http://www.deja.com/



Thu, 10 Jul 2003 09:38:06 GMT  
 sed question: unreliable "/^$/d" ?

Quote:

> Who ever said Drew Krause couldn't write what follows?:
>>Hello,
>>I hop this question would still interest the group, though it concerns
>>sed. I have a large (500+) group of text files that my unix system isn't
>>reading well, mostly errors in spaces, blank lines etc. The /^$/d
>>command doesn't seems to snag the blank lines -- has anyone else had
>>this problem?

>>Any suggestions appreciated -- thanks!

>>Drew Krause

> Make sure there are no tabs or spaces in the blank lines. I think the regexp
> "/^[:space:]$/" ought to fix that. I'm not quite sure if sed handles the [::]
> style regexp's.

Howzabout /^\s*$/d (\s should match spaces tabs etc).
-ian.

Quote:
> --
> Bryan Hoyt

> http://www.crosswinds.net/~artmusic
> -----------------------------------
> Old musicians never die, they just decompose.

--


Tue, 15 Jul 2003 01:21:43 GMT  
 
 [ 10 post ] 

 Relevant Pages 

1. beginners "let"/"random" question

2. sed - extracting "*.cpp" from a file

3. Looking for MS-DOS "sed"

4. Trouble with "exec sed" on NT.

5. string.join(["Tk 4.2p2", "Python 1.4", "Win32", "free"], "for")

6. BEGIN{want[]={"s1o", "s2o", "s2q", "s3q"}

7. Parsing ""D""?

8. "Fifth", "Forth", zai nar?

9. MicroWorlds Pro "QUESTION Set Size" question

10. Ruby "finalize", "__del__"

11. ANNOUNCE: new "plus"- and "dash"-patches available for Tcl7.5a2/Tk4.1a2

12. Looking for "stdin", "stdout"

 

 
Powered by phpBB® Forum Software