Most efficient sort text file routine, sort algorithms 
Author Message
 Most efficient sort text file routine, sort algorithms

I have a text file. Each line begins with a date like "19970515".
There might be up to a few hundred dates in this text file, possibly
in random order, but I cannot be sure
someone might not put thousands of lines in there.

I am avoiding a database with an index, because
that way users cannot edit the text file with Notepad.exe or other
text editor.

File example:

19970514 This is a record
19970521 This is more info
19961212 This is some data
19980202 This is another record
19950608 This is yet more info

I am looking for a clever way to sort this file in the most efficient
manner. Each line is less than 255 characters.

So I am thinking:

 TStringList.loadFromFile;
 TStringList.sort;
 TStringList.saveToFile;

Using delphi 16-bit, I wonder, what are the limits to the number of
strings?  If the limit is 64K, and 200 bytes per
line, then 5 records per K, 60*5 +4*5 = 300+20= 320 records
max. That would not be enough.

I seem to remember some old-fashioned sort algorithm. Two files would
run against each other with some sort of buffer where records are
compared, but don't recall how this is done without scanning and
re-reading the entire
file countless numbers of times.

Email me. I reply to every email.
Matthew



Mon, 01 Nov 1999 03:00:00 GMT  
 Most efficient sort text file routine, sort algorithms

Quote:

>I have a text file. Each line begins with a date like "19970515".
>There might be up to a few hundred dates in this text file, possibly
>in random order, but I cannot be sure
>someone might not put thousands of lines in there.
>I am avoiding a database with an index, because
>that way users cannot edit the text file with Notepad.exe or other
>text editor.
>File example:
>19970514 This is a record
>19970521 This is more info
>19961212 This is some data
>19980202 This is another record
>19950608 This is yet more info
>I am looking for a clever way to sort this file in the most efficient
>manner. Each line is less than 255 characters.
>So I am thinking:
> TStringList.loadFromFile;
> TStringList.sort;
> TStringList.saveToFile;
>Using Delphi 16-bit, I wonder, what are the limits to the number of
>strings?  If the limit is 64K, and 200 bytes per
>line, then 5 records per K, 60*5 +4*5 = 300+20= 320 records
>max. That would not be enough.
>I seem to remember some old-fashioned sort algorithm. Two files would
>run against each other with some sort of buffer where records are
>compared, but don't recall how this is done without scanning and
>re-reading the entire
>file countless numbers of times.
>Email me. I reply to every email.
>Matthew

Umm... TurboPower Software sells a great sort engine as part of its SysTools
package.  Generally speaking the best way to get a large-scale sort done is to
buy a package to do it.  Seriously.  I mean that very seriously!


Tue, 02 Nov 1999 03:00:00 GMT  
 Most efficient sort text file routine, sort algorithms

Quote:


> >I have a text file. Each line begins with a date like "19970515".
> >There might be up to a few hundred dates in this text file, possibly
> >in random order, but I cannot be sure
> >someone might not put thousands of lines in there.

> >I am avoiding a database with an index, because
> >that way users cannot edit the text file with Notepad.exe or other
> >text editor.

> >File example:

> >19970514 This is a record
> >19970521 This is more info
> >19961212 This is some data
> >19980202 This is another record
> >19950608 This is yet more info

> >I am looking for a clever way to sort this file in the most efficient
> >manner. Each line is less than 255 characters.

> >So I am thinking:

> > TStringList.loadFromFile;
> > TStringList.sort;
> > TStringList.saveToFile;

> >Using Delphi 16-bit, I wonder, what are the limits to the number of
> >strings?  If the limit is 64K, and 200 bytes per
> >line, then 5 records per K, 60*5 +4*5 = 300+20= 320 records
> >max. That would not be enough.

> >I seem to remember some old-fashioned sort algorithm. Two files would
> >run against each other with some sort of buffer where records are
> >compared, but don't recall how this is done without scanning and
> >re-reading the entire
> >file countless numbers of times.

> >Email me. I reply to every email.
> >Matthew

> Umm... TurboPower Software sells a great sort engine as part of its SysTools
> package.  Generally speaking the best way to get a large-scale sort done is to
> buy a package to do it.  Seriously.  I mean that very seriously!

Turbo Power used to give away their Pascal sort (MSortP), works great.
Yo should be able to port it to Delphi.
--

Robert

Return address changed to deter malevolent spammers.



Wed, 03 Nov 1999 03:00:00 GMT  
 
 [ 3 post ] 

 Relevant Pages 

1. Efficient sorting algorithm?

2. Shell sort vs. Comb sort (was: sorting arrays)

3. Program to sort real values in a text file

4. SORTING RECORDS-NON TEXT FILES

5. Help: Sorting a text file

6. Counting Comparisons in a Sorting Algorithm

7. Help! sorting algorithm

8. Quick Sort Algorithm or Source Code

9. Independent Type Algorithm (Eg: Sorting)

10. sort algorithm trouble...

11. algorithm to calculate the median using a sorted array

12. algorithm to calculate the median for a sorted array

 

 
Powered by phpBB® Forum Software