How to split file on max number of lines 
Author Message
 How to split file on max number of lines

Hello-

I'm trying to emulate the unix split function on a windows platform,
and I'm having problems. Here's what I want to do:

My script is generating a text file of variable length. The resulting
text file can be no longer than 20,000 lines. If it is longer, I need
to split it accordingly (that is, so that each piece is less than
20,000 lines). I'm kind of struggling to make that happen. I think
that incremementing a counter is the way, but its not working out.
This is what I have so far:

#This is where i count the number of lines in file
$lines = 0;
      open(FILE, $to_be_counted) or die "Can't open $to_be_counted:
$!";
      while (sysread FILE, $buffer, 4096) {
          $lines += ($buffer =~ tr/\n//);
      }
    close FILE;

    if ($lines > 20000) {
       $count = 0;
       $file_num = 0;
          open(FILE2, $to_be_counted) or die "Can't open file for
split: $!";
              while (defined ($buffer = <FILE2>)) {
                  if ($count =< 20000) {
                      print SOME_OUTPUT_FILEHANDLE $buffer;
                      $count++;
                         } else {
                       $count = 0; #set the count back to zero
                       $file_num++; #increment the outfile name
                       print SOME_NEW_OUTPUTFILENAME $buffer;
                       }
                  }

I guess my biggest problem is how to do the printing to the filename
without opening a new file handle on each pass (which I would think
would be kind of I/O intensive). In a perfect world, at the end of the
splitting I would have file1.txt, file2.txt, file3.txt, with all being
less than 20K lines.

Any ideas? Am I even on the right track? Is there some Dos system call
that I could cheat and use?

Thanks!

David Kenneally



Sun, 16 May 2004 09:09:37 GMT  
 How to split file on max number of lines

  DK> I'm trying to emulate the unix split function on a windows platform,
  DK> and I'm having problems. Here's what I want to do:

here is a pure perl version already done:

        http://www.perl.com/language/ppt/src/split/index.html

uri

--

-- Stem is an Open Source Network Development Toolkit and Application Suite -
----- Stem and Perl Development, Systems Architecture, Design and Coding ----
Search or Offer Perl Jobs  ----------------------------  http://jobs.perl.org



Sun, 16 May 2004 09:12:54 GMT  
 
 [ 2 post ] 

 Relevant Pages 

1. Max number of lines in Tk::Text

2. Max number of open files?

3. Max number of open filehandles in forking process?

4. max international, mlm business opportunity, MaxGXL, network marketing opportunity , Max GXL, MaxWLX, Max N-fuze, Max WLX,

5. Max number of options in GetOptions

6. Splitting file at empty line

7. DOS Perl: max input line length?

8. Max lines in an Array?

9. Read and Split Individual Text File Lines

10. splitting lines of DOS file in UNIX

11. Splitting a line from the log file.

 

 
Powered by phpBB® Forum Software