DB_File file size limit under Windows NT 4.0? 
Author Message
 DB_File file size limit under Windows NT 4.0?

Does anyone know if it's possible to create database files larger than
2GB using Berkeley DB (v. 3.3.11) via the DB_File module (v. 1.73)
under Windows NT 4.0 (NTFS file system)?  I've successfully created
larger flat files than this before, so NT is not the problem.  The
Berkeley DB documentation indicates it can handle humungous files, and
the DB_File documentation does not discuss size limits.

However, it was the case that when I installed Perl (ActivePerl
5.6.1.626 from http://www.*-*-*.com/ ), the documentation for
DB_File only listed Linux support, despite my having downloaded a
binary distribution for Windows NT.  Nor was DB_File included in the
distribution.  However, some old postings on this group encouraged me
to try using DB_File anyway.  I downloaded Berkeley DB, compiled it
with Visual C++ 6.0, used ppm to download DB_File, and until running
into this file size limit I've had no problems with it.

So maybe DB_File, or the version I have, is assuming a 2GB Unix file
size limit? If so, is there a way to change this? Any other advice?

Thanks, Dave



Mon, 12 Apr 2004 03:46:11 GMT  
 DB_File file size limit under Windows NT 4.0?
Quote:

> Does anyone know if it's possible to create database files larger than
> 2GB using Berkeley DB (v. 3.3.11) via the DB_File module (v. 1.73)
> under Windows NT 4.0 (NTFS file system)?  I've successfully created
> larger flat files than this before, so NT is not the problem.  The
> Berkeley DB documentation indicates it can handle humungous files, and
> the DB_File documentation does not discuss size limits.

> However, it was the case that when I installed Perl (ActivePerl
> 5.6.1.626 from http://www.activestate.com), the documentation for
> DB_File only listed Linux support, despite my having downloaded a
> binary distribution for Windows NT.  Nor was DB_File included in the
> distribution.  However, some old postings on this group encouraged me
> to try using DB_File anyway.  I downloaded Berkeley DB, compiled it
> with Visual C++ 6.0, used ppm to download DB_File, and until running
> into this file size limit I've had no problems with it.

> So maybe DB_File, or the version I have, is assuming a 2GB Unix file
> size limit? If so, is there a way to change this? Any other advice?

> Thanks, Dave

This may be an issue with how the AS perl was compiled.  When manually
compiling, the config script asks if support for large files is
required.
--
Ron Reidy
Oracle DBA
Reidy Consulting, L.L.C.


Mon, 12 Apr 2004 06:08:01 GMT  
 DB_File file size limit under Windows NT 4.0?

Quote:

> This may be an issue with how the AS perl was compiled.  When manually
> compiling, the config script asks if support for large files is
> required.

Ron - OK, that's a possibility, since I downloaded the precompiled
version.  Is there a way to check this before I look into what would
be involved in getting source and compiling?

Thanks, Dave



Mon, 12 Apr 2004 11:03:32 GMT  
 DB_File file size limit under Windows NT 4.0?
Quote:


> > This may be an issue with how the AS perl was compiled.  When manually
> > compiling, the config script asks if support for large files is
> > required.

> Ron - OK, that's a possibility, since I downloaded the precompiled
> version.  Is there a way to check this before I look into what would
> be involved in getting source and compiling?

> Thanks, Dave

perl -V

Look for "uselargefiles=define" under the Platform section.
--
Ron Reidy
Oracle DBA
Reidy Consulting, L.L.C.



Mon, 12 Apr 2004 11:13:33 GMT  
 DB_File file size limit under Windows NT 4.0?
Ron Reidy <:">

Quote:



>> > This may be an issue with how the AS perl was compiled.
>> > When manually compiling, the config script asks if support
>> > for large files is required.

>> Ron - OK, that's a possibility, since I downloaded the
>> precompiled version.  Is there a way to check this before I
>> look into what would be involved in getting source and
>> compiling?

>> Thanks, Dave
> perl -V

> Look for "uselargefiles=define" under the Platform section.

My AS perl 5.6.1.629 says undef. Looks like it says undef for alot
of things in there.

--
Rob - http://rock13.com/
Web Stuff: http://rock13.com/webhelp/



Mon, 12 Apr 2004 15:14:25 GMT  
 DB_File file size limit under Windows NT 4.0?

Quote:



> > > This may be an issue with how the AS perl was compiled.  When manually
> > > compiling, the config script asks if support for large files is
> > > required.

> > Ron - OK, that's a possibility, since I downloaded the precompiled
> > version.  Is there a way to check this before I look into what would
> > be involved in getting source and compiling?

> > Thanks, Dave
> perl -V

> Look for "uselargefiles=define" under the Platform section.

Ron - Thanks, I do get uselargefiles=undef, alas.  OK, guess I'll bite
the bullet and try compiling from source.  I don't see a source
distribution at activestate.com, so I got the latest development
version from CPAN.  I gather from your comment that this "config
script" will be run during the compile. Do you know if there's any
changes that need to be made in the Makefile or in the settings of the
compiler (Visual C++ 6.0 in my case)?

Dave



Tue, 13 Apr 2004 12:12:09 GMT  
 DB_File file size limit under Windows NT 4.0?


Quote:





>> > > This may be an issue with how the AS perl was compiled.  When
>> > > manually compiling, the config script asks if support for large
>> > > files is required.

>> > Ron - OK, that's a possibility, since I downloaded the precompiled
>> > version.  Is there a way to check this before I look into what would
>> > be involved in getting source and compiling?

>> > Thanks, Dave
>> perl -V

>> Look for "uselargefiles=define" under the Platform section.

> Ron - Thanks, I do get uselargefiles=undef, alas.  OK, guess I'll bite
> the bullet and try compiling from source.  I don't see a source
> distribution at activestate.com, so I got the latest development
> version from CPAN.  I gather from your comment that this "config
> script" will be run during the compile. Do you know if there's any
> changes that need to be made in the Makefile or in the settings of the
> compiler (Visual C++ 6.0 in my case)?

Just try to go for the obvious :) and use -Duselargefiles
You might consider -Duse64bitint, -Duse64bitall and the
-Dusethreads -Duseithreads -Ubin5005compat combination

# gzip -d < perl-5.6.1.tar.gz | tar xvf -
# cd perl-5.6.1
# sh Configure -Duselargefiles -des
:
:
# make
:
:
# make test
:
:
# make install

From the INSTALL file:

=head2 Large file support.

Since Perl 5.6.0 Perl has supported large files (files larger than
2 gigabytes), and in many common platforms like Linux or Solaris this
support is on by default.

This is both good and bad. It is good in that you can use large files,
seek(), stat(), and -s them.  It is bad if you are interfacing Perl
using some extension, also the components you are connecting to must
be large file aware: if Perl thinks files can be large but the other
parts of the software puzzle do not understand the concept, bad things
will happen.  One popular extension suffering from this ailment is the
Apache extension mod_perl.

There's also one known limitation with the current large files
implementation: unless you also have 64-bit integers (see the next
section), you cannot use the printf/sprintf non-decimal integer
formats like C<%x> to print filesizes.  You can use C<%d>, though.

--
H.Merijn Brand    Amsterdam Perl Mongers (http://www.amsterdam.pm.org/)
using perl-5.6.1, 5.7.1 & 628 on HP-UX 10.20 & 11.00, AIX 4.2, AIX 4.3,
     WinNT 4, Win2K pro & WinCE 2.11 often with Tk800.022 &/| DBD-Unify
ftp://ftp.funet.fi/pub/languages/perl/CPAN/authors/id/H/HM/HMBRAND/



Tue, 13 Apr 2004 18:07:45 GMT  
 DB_File file size limit under Windows NT 4.0?
[snip]

Quote:
> There's also one known limitation with the current large files
> implementation: unless you also have 64-bit integers (see the next
> section), you cannot use the printf/sprintf non-decimal integer
> formats like C<%x> to print filesizes.  You can use C<%d>, though.

How about the following, or some variant thereof?
print unpack("H*", pack("q", -s "file")), "\n";

--
Klein bottle for rent - inquire within.



Thu, 15 Apr 2004 13:35:26 GMT  
 
 [ 8 post ] 

 Relevant Pages 

1. fixes for file size limits with DB_File/Berkeley DB

2. Need to know how to somehow limit file sizes in file uploads with Perl

3. Need to know how to limit the file sizes in file uploads in Perl

4. CGI.pm Bug: file upload file size mismatch when file size is close to buffer size

5. file size limits of perl Net::ftp

6. file size limit

7. Net::FTP File Size Limit?

8. ?? Size limit on files/arrays??

9. limited tuple size in DBM files?

10. Size limit when uploading file ?

11. File Handle Data size limits

12. limited tuple size in DBM files?

 

 
Powered by phpBB® Forum Software