Segmented Mac OS X dev tools (was: Yet another gcc on Mac OS X posting) 
Author Message
 Segmented Mac OS X dev tools (was: Yet another gcc on Mac OS X posting)

[ Note: In our last episode, a sometime programmer (me) attemped to
  segment the PublicBeta_Developer.tgz into several smaller bits for
  easier download. A perl script was employed to segment them, and
  'cat' to try to put them back together. See http://www.*-*-*.com/
  for details.  Script in question included at end of post for those who
don't    
  want to go to the URL. Summary included for new cross-post to  
  comp.lang.perl.misc. ]

Quote:
> >It doesn't work, though, and I'm not quite sure why. After I
> >reassemble the files I get the following:


> >gunzip: PB_Dev.reassembled.tar.gz: unexpected end of file

> >Hmmmm. Any ideas why?

> Assuming your perl script is not broken, I would assume that it's
> because .gz files are binary, but cat is treating them as text.

I suppose this is as good an explanation as any, but I'd always thought
that on unix systems there is no difference between text and binary data
(ie printable and unprintable bytes, but handling text bytes and data bytes
were the same).

Quote:
> Your
> script may also be treating them as text.

I'm trying to treat the data as binary in the perl script; I didn't
invoke BINMODE, but I'm using perl's read() function, which grabs a
specified number of bytes rather than a line of text. Again, I'm not
sure this even matters: from what I've read and been told, there is
no text/binary distinction in perl except on Windows/DOS. But then again,
every once in a while, this issue jumps up and bites me (I've still
never gotten HTTP upload to work properly for binary files using perl).

The other thing that seems funny to me: I wrote another script to segment
a uuencoded file, which did deal with lines of text, which I thought might
be OK since uuencodedstuff is all printable... ended up with exactly the
same problem as I described above.

Oh yes, the one other funny thing: the fact that the original, unsegmented
gzipped archive somehow becomes unreadable by gzip after I pass it through
this perl script, despite the fact that as far as I can tell, I'm opening
the .gz for read only.

[c.p.l.m folks: nothing more perl related below]

Quote:
> Before you do too much, see
> if your shell server supports resuming FTP, which will save you from
> having to do anything.

Would this happen automatically if it did support it? If not, how would
I check, or make this happen?

Quote:
> Failing that, I wouldn't be surprised if tar had
> built-in support for segmenting and unsegmenting files (but I don't
> really know).

I'll try again to understand the man page. There is a provision for
specifying block size for tar, but I can't tell exactly how to make this work.
Thanks for the idea, though.

Quote:
> I would imagine there must be
> a utility to deal with this already there.

Anyone know of anything else?

=================================================================
"The best laid plans of mice and men are about equal."

Address is spam repelant. Remove eights to reach me.



Sun, 22 Jun 2003 10:06:09 GMT  
 Segmented Mac OS X dev tools (was: Yet another gcc on Mac OS X posting)


Wed, 18 Jun 1902 08:00:00 GMT  
 Segmented Mac OS X dev tools (was: Yet another gcc on Mac OS X posting)
On Wed, 03 Jan 2001 02:06:09 GMT,

[no attributions present]

Quote:
>> >Hmmmm. Any ideas why?

>> Assuming your perl script is not broken, I would assume that it's
>> because .gz files are binary, but cat is treating them as text.

cat shouldn't care.

Quote:
>> Your
>> script may also be treating them as text.

> I'm trying to treat the data as binary in the perl script; I didn't
> invoke BINMODE, but I'm using perl's read() function, which grabs a

You should. Even if you are on a platform where you think it doesn't
matter, you should always, always use binmode if you mean to read a
binary file. If not for portability, at least for code documentation.
Just use it.

The documentation for binmode got changed quite a bit to make sure that
this point was made clearly.

Quote:
> specified number of bytes rather than a line of text. Again, I'm not
> sure this even matters: from what I've read and been told, there is
> no text/binary distinction in perl except on Windows/DOS. But then again,

That isn't true. There are many other platforms where there is a
distinction. But you shouldn't worry about that, or try to guess it. If
you have binary data, you use binmode.

Quote:
> every once in a while, this issue jumps up and bites me (I've still
> never gotten HTTP upload to work properly for binary files using perl).

Are you using the CGI.pm module?

Quote:
> The other thing that seems funny to me: I wrote another script to segment
> a uuencoded file, which did deal with lines of text, which I thought might
> be OK since uuencodedstuff is all printable... ended up with exactly the
> same problem as I described above.

Sounds like you're doing something else wrong then. Maybe you're missing
a byte here or there?

Quote:
> Oh yes, the one other funny thing: the fact that the original, unsegmented
> gzipped archive somehow becomes unreadable by gzip after I pass it through
> this perl script, despite the fact that as far as I can tell, I'm opening
> the .gz for read only.

I find that terribly hard to believe. If you open something for read,
there is simply no way that it gets modified.

Quote:
> [c.p.l.m folks: nothing more perl related below]

Thanks for the warning.

I don't carry the other groups on my news server, so I can't go back to
find the thread that started all this.  The URL you posted doesn't let
me have a look at the scripts in that directory, so there won't be any
really Perl (or shell) specific help here. Maybe what you should do is
post your code (it's only 1 kb after all).

It's impossible to say anything sensible about what your scripts do
without seeing them. Reconfigure your web server, or rename the files,
or post the stuff.

Martien
--
Martien Verbruggen              |
Interactive Media Division      | Useful Statistic: 75% of the people
Commercial Dynamics Pty. Ltd.   | make up 3/4 of the population.
NSW, Australia                  |



Sun, 22 Jun 2003 10:30:36 GMT  
 Segmented Mac OS X dev tools (was: Yet another gcc on Mac OS X posting)


Wed, 18 Jun 1902 08:00:00 GMT  
 Segmented Mac OS X dev tools (was: Yet another gcc on Mac OS X posting)
In article


[other stuff snipped]

Quote:
>> Before you do too much, see
>> if your shell server supports resuming FTP, which will save you from
>> having to do anything.

>Would this happen automatically if it did support it? If not, how would
>I check, or make this happen?

You need both a server and a client that support resume. Anarchie or
whatever it's being called these days has good support, as does
Transmit. Fetch supports it to an extent, but IIRC you can't resume a
file if you've quit (or crashed) Fetch since starting the download.

As for figuring out if server support is there, use one of the
aforementioned programs and start getting a file, then kill it right
away and see if it has to start over or not.

--
"Our doubts are traitors, and make us lose the good we oft might win by fearing to attempt." - William Shakespeare

Mike Ash - www.mikeash.com   My reply address is valid, it has no spamproofing.



Mon, 23 Jun 2003 02:53:06 GMT  
 Segmented Mac OS X dev tools (was: Yet another gcc on Mac OS X posting)
On Wed, 3 Jan 2001 10:53:06 -0800, Michael Ash wrote

Quote:
> As for figuring out if server support is there, use one of the
> aforementioned programs and start getting a file, then kill it right
> away and see if it has to start over or not.

Has anyone noticed in "Transmit" is says something like:

Host:  Unix (Resume OK)
Host:  Mac (Resume OK)
Host:  Windows NT

I've never seen "Resume OK" under Windows NT, anyone know why? Doesn't it
support resume? Ot is it that Transmit doesn't support Resume when talking to
an NT Box?

Thanks a lot
Dave



Mon, 23 Jun 2003 06:21:53 GMT  
 Segmented Mac OS X dev tools (was: Yet another gcc on Mac OS X posting)


Wed, 18 Jun 1902 08:00:00 GMT  
 Segmented Mac OS X dev tools (was: Yet another gcc on Mac OS X posting)
[A complimentary Cc of this posting was NOT sent to Martien Verbruggen


Quote:
> >> Assuming your perl script is not broken, I would assume that it's
> >> because .gz files are binary, but cat is treating them as text.

> cat shouldn't care.

Of course it would.  Remember C-z?

Quote:
> > I'm trying to treat the data as binary in the perl script; I didn't
> > invoke BINMODE, but I'm using perl's read() function, which grabs a

> You should. Even if you are on a platform where you think it doesn't
> matter, you should always, always use binmode if you mean to read a
> binary file. If not for portability, at least for code documentation.
> Just use it.

> The documentation for binmode got changed quite a bit to make sure that
> this point was made clearly.

Yes.  AFAIK, binmode/textmode (=newline translation) is orthogonal to
(non)buffering (which is what read() is about) *on all the Perl platforms*.

Ilya



Mon, 23 Jun 2003 07:39:40 GMT  
 Segmented Mac OS X dev tools (was: Yet another gcc on Mac OS X posting)


Wed, 18 Jun 1902 08:00:00 GMT  
 Segmented Mac OS X dev tools (was: Yet another gcc on Mac OS X posting)

Quote:

>Oh yes, the one other funny thing: the fact that the original, unsegmented
>gzipped archive somehow becomes unreadable by gzip after I pass it through
>this perl script, despite the fact that as far as I can tell, I'm opening
>the .gz for read only.

My guess is that, in that case, it's really that the "original" archive is
*also* a short read.  Look at the "master" copy you started with and make
sure it's the same size.

Drop me a line with the URL you were getting the original from, and I'll see
if it looks like a complete archive to me.

-s
--

C/Unix wizard, Pro-commerce radical, Spam fighter.  Boycott Spamazon!
Consulting & Computers: http://www.plethora.net/



Tue, 24 Jun 2003 11:41:26 GMT  
 Segmented Mac OS X dev tools (was: Yet another gcc on Mac OS X posting)


Wed, 18 Jun 1902 08:00:00 GMT  
 Segmented Mac OS X dev tools (was: Yet another gcc on Mac OS X posting)

Quote:

>[A complimentary Cc of this posting was NOT sent to Martien Verbruggen


>> >> Assuming your perl script is not broken, I would assume that it's
>> >> because .gz files are binary, but cat is treating them as text.
>> cat shouldn't care.
>Of course it would.  Remember C-z?

"cat" is a Unix program, and on Unix, ^Z is just another character that
can be part of any file, at any point in the file.  (Or at least, the OP
was talking about using it on Unix.)

Quote:
>Yes.  AFAIK, binmode/textmode (=newline translation) is orthogonal to
>(non)buffering (which is what read() is about) *on all the Perl platforms*.

I don't know that canonical Unix perl does any "newline translation".  There's
no native reason to.

-s
--

C/Unix wizard, Pro-commerce radical, Spam fighter.  Boycott Spamazon!
Consulting & Computers: http://www.plethora.net/



Tue, 24 Jun 2003 11:43:22 GMT  
 Segmented Mac OS X dev tools (was: Yet another gcc on Mac OS X posting)
[A complimentary Cc of this posting was sent to Peter Seebach


Quote:
> >> >> Assuming your perl script is not broken, I would assume that it's
> >> >> because .gz files are binary, but cat is treating them as text.

> >> cat shouldn't care.

> >Of course it would.  Remember C-z?

> "cat" is a Unix program

???  What makes a program a "Unix program"?

, and on Unix, ^Z is just another character that

Quote:
> can be part of any file, at any point in the file.  (Or at least, the OP
> was talking about using it on Unix.)

If latter is true, then binmode is a NOP, so the whole discussion
would be void.

Ilya



Tue, 24 Jun 2003 12:45:48 GMT  
 Segmented Mac OS X dev tools (was: Yet another gcc on Mac OS X posting)

MCMXCIII in <URL::">
~~ [A complimentary Cc of this posting was sent to Peter Seebach


~~ > >> >> Assuming your perl script is not broken, I would assume that it's
~~ > >> >> because .gz files are binary, but cat is treating them as text.
~~ >
~~ > >> cat shouldn't care.
~~ >
~~ > >Of course it would.  Remember C-z?
~~ >
~~ > "cat" is a Unix program
~~
~~ ???  What makes a program a "Unix program"?

History.

Abigail
--
$_ = "\nrekcaH lreP rehtona tsuJ"; my $chop; $chop = sub {print chop; $chop};
$chop -> () -> () -> () -> () -> () -> () -> () -> () -> () -> () -> () -> ()
-> () -> () -> () -> () -> () -> () -> () -> () -> () -> () -> () -> () -> ()



Tue, 24 Jun 2003 19:31:25 GMT  
 Segmented Mac OS X dev tools (was: Yet another gcc on Mac OS X posting)

Quote:

}[A complimentary Cc of this posting was sent to Peter Seebach


}> >> >> Assuming your perl script is not broken, I would assume that it's
}> >> >> because .gz files are binary, but cat is treating them as text.
}>
}> >> cat shouldn't care.
}>
}> >Of course it would.  Remember C-z?
}>
}> "cat" is a Unix program
}
}???  What makes a program a "Unix program"?
}
}, and on Unix, ^Z is just another character that
}> can be part of any file, at any point in the file.  (Or at least, the OP
}> was talking about using it on Unix.)
}
}If latter is true, then binmode is a NOP, so the whole discussion
}would be void.

Binmode on nearly all Unix systems (including OS X) _IS_ a no-op.
--

"Extremism in defense of liberty is no vice, and moderation in pursuit
of justice is no virtue."


Wed, 25 Jun 2003 03:12:25 GMT  
 
 [ 19 post ]  Go to page: [1] [2]

 Relevant Pages 

1. Converting Mac line terminator to Unix terminator on Mac OS X

2. Device::SerialPort on Mac OS-X

3. perl GUIs under mac os x

4. Perl and Mac Os X,

5. Perl on Mac OS 8.5?

6. Problems with CPAN on Mac OS X

7. Run perl script under Mac OS X

8. Mac OS X and Perl 6

9. Mac OS X and mod_perl - mystery

10. 5.8.0 Perl compile problem on Mac OS 10.1.5

11. Sys::Syslog & Mac OS X

12. Convert .pl to .exe for Win32 in Mac OS X

 

 
Powered by phpBB® Forum Software