Hello again,
I'm still working on this same project, and I've recently hit a major
conceptual snag. This deals with perl in that perl is what the project
itself is written in, so this may not be the place to ask, but:
The project stores the bulk of its data (user-input updates / journal
entries / what-have-you) in distinct files, each of which represents
the happenings on one specific date, as given by the filename.
In other words, the file 19990101 contains the entry for 01 Jan 1999;
19990102 is 02 Jan 1999, etc.
The problem is this: I find it necessary to take some data from each
file in order to give the user useful information about these files. I
liken this to be roughly akin to a rudimentary search engine.
The problem is, I would like to avoid building a "lookup table" that
contains all of the relevant information about all entries, because
this would be an inefficient use of disk space.
However, since the entire thing is being accessed from the web, parsing
through all of the files to retrieve the data is an inefficient use of
processor time and may cause unnecessary wear and tear on the webhost's
disks.
To give some idea as to the scope of the problem, we're talking about a
maximum of 1000 entries (roughly three years of writing every day, or
five years writing every other), and entries with an average of, say,
5K (my average is 2K).
Worst case, we're talking about 5MB of files. If, say, 100 people are
reading on a daily basis (uh, yeah right, but anyway), then the actual
data the script is sifting through shoots to 5GB.
Now granted, this is all worst-case scenario; there are things that
would cut the data sifting in half or even almost eliminate it that I'm
not taking into consideration.
The point is, how efficient is perl's filehandling? Would it show an
obvious performance hit on the webserver if I were opening and closing
files repeatedly? No writes would be done, only reads.
I don't think it's possible to go to a database solution, which would
be the logical choice.
Is it possible to run a CGI script as a daemon? Would that be
applicable here?
--
-Stephen Deken, type designer and general geek
Sent via Deja.com http://www.*-*-*.com/
Before you buy.