Fujitsu COBOL 4.2 ISAM Questions 
Author Message
 Fujitsu COBOL 4.2 ISAM Questions

Hello All,

We have a project coming up that will require loading approximately 81
million 300 byte records into a file.  This file will then be searched
using a file from our mainframe as input.  The accesses necessary will
be 1. Name 2. SSN.

Initial thought is to use ISAM since the indexes are straight forward
and I can build and search the file very easily directly from COBOL
without any external database software involved - plus I have the
search routines already written in COBOL on the mainframe.  The file
is to be loaded, searched then removed.  This process would occur
quarterly.  We are attempting to update addresses in our mainframe
database with the more current addresses in this file.  Building the
file on the mainframe is out - too costly to add the necessary
disk/processor upgrade.  NT server would be much cheaper to put
together and quicker!  After the match process the results would be
used to update the mainframe database (estimate a 10% match rate).

1.  Can an ISAM file handle 81 million records?  Are there any
limitations (other than the obvious like disk space) for ISAM files?

2.  I was told that Fujitsu ISAM files have the keys embedded in the
main data file.  MF, I know, has a separate file for the indexes.  Is
this true?

3.  I read through the Fujitsu documentation on the File Utility and
it appears to me that I am going to need at least an equal amount of
disk as the file I'm making into an ISAM - probably more like 2.5 - 3
times, for the output and work files.  True?  

4.  Assuming ISAM is out of the question, what would be your best
guess/opinion for an alternative file/database format?  Keep in mind
that I have VERY limited experience interfacing COBOL to anything else
(ODBC for example, minimal API and usually I get very {*filter*}y in the
process :-0).

Thanks for your opinions and assistance.

Doug Blaze



Sun, 19 May 2002 03:00:00 GMT  
 Fujitsu COBOL 4.2 ISAM Questions

Quote:
Blaze) writes:
>file on the mainframe is out - too costly to add the necessary
>disk/processor upgrade.  NT server would be much cheaper to put
>together and quicker!  After the match process the results would be
>used to update the mainframe database (estimate a 10% match rate).

>1.  Can an ISAM file handle 81 million records?  Are there any
>limitations (other than the obvious like disk space) for ISAM files?

>2.  I was told that Fujitsu ISAM files have the keys embedded in the
>main data file.  MF, I know, has a separate file for the indexes.  Is
>this true?

>3.  I read through the Fujitsu documentation on the File Utility and
>it appears to me that I am going to need at least an equal amount of
>disk as the file I'm making into an ISAM - probably more like 2.5 - 3
>times, for the output and work files.  True?  

>4.  Assuming ISAM is out of the question, what would be your best
>guess/opinion for an alternative file/database format?  Keep in mind
>that I have VERY limited experience interfacing COBOL to anything else
>(ODBC for example, minimal API and usually I get very {*filter*}y in the
>process :-0).

>Thanks for your opinions and assistance.

>Doug Blaze

I think that your first problem is going to be finding a compiler/file system
that will permit files > 2G.  I guestimate that  your 81M*300 is going to
be equivalent to 24300M or 24.3G.  Are you sure that there is not a method
that you could use a record number and name component in a lookup/match
file process?  We handle large files on TAPE/CART.  Our processes usually
strip the key match fields and a sequence (to get back to the original record)
from the two main files, sort these extracts by the match key, match by match
key and collect the sequence of the matching record (writing only those that
matched).  Use this file to go back and pull the remaining data from the new
data file
appending the master seq.  Sort the newdata extract by the master seq.
finally use a sequential match program to read the master and compare the
master seq to the master seq of the new data extract.

Granted that this is a long round about way to go , but in our environment it
is
the best we can do.   There is nothing out there right now that could hanlde a
24G file on the PC that I am aware of.  You might be able to break the main
file
into smaller files by first letter , but you are still looking at the
requirement of
almost 100G of space for the flat file and its many smaller ISAM files.
This does not even consider the size of your master fiile that is to be
updated.



Sun, 19 May 2002 03:00:00 GMT  
 Fujitsu COBOL 4.2 ISAM Questions

Quote:

>file on the mainframe is out - too costly to add the necessary
>disk/processor upgrade.  NT server would be much cheaper to put
>together and quicker!  After the match process the results would be
>used to update the mainframe database (estimate a 10% match rate).

That seems unlikely.  The bigger the file, the more cost effective it is
to add to the mainframe rather than an NT machine.  Unless you have an
adequate NT machine sitting around doing nothing that is...

Sort of like having a truck load of data and deciding to save money by
adding a bunch of trailers to a sports car to carry it across the
country!



Sun, 19 May 2002 03:00:00 GMT  
 Fujitsu COBOL 4.2 ISAM Questions

Quote:

(Doug
> Blaze) writes:
[snip]
> >1.  Can an ISAM file handle 81 million records?  Are there any
> >limitations (other than the obvious like disk space) for ISAM files?

According to the Fujitsu v4 docs, file size is limited to 1 gig.

See below...

Quote:

> >2.  I was told that Fujitsu ISAM files have the keys embedded in the
> >main data file.  MF, I know, has a separate file for the indexes.  Is
> >this true?

I believe it is true that Fujitsu indexed files contain the key trees and
data in the same file.

Quote:

> >3.  I read through the Fujitsu documentation on the File Utility and
> >it appears to me that I am going to need at least an equal amount of
> >disk as the file I'm making into an ISAM - probably more like 2.5 - 3
> >times, for the output and work files.  True?

Depends on your data compression factor.

[snip]

Quote:
> >Thanks for your opinions and assistance.
> >Doug Blaze

> I think that your first problem is going to be finding a compiler/file
system
> that will permit files > 2G.  I guestimate that  your 81M*300 is going to
> be equivalent to 24300M or 24.3G.  [snip alternative method]

Certainly 24G is not a problem for NT.  Drives are readily available to hold
30+G.  RAID systems will take you above this.

RM/COBOL file system will support  this (up to terabytes) on NT (which Doug
indicated is the platform) using the NT File System.

RM/COBOL supports both data and key compression so it is possible that your
file might not hit this size.  Key compression reduces index tree heights as
well as reducing the space used for the tree.  Both names and SSN are very
good candidates for key compression.  If the data records are name/addresses
and other text fields, data compression should also be very effective in
reducing the amount of disk space used for each record.

While RM/COBOL is not "free", we do have a 30-day money-back guarantee.
There is no risk to try it out.  Call 1-800-RM-COBOL (762-6265).

Best regards,
Tom Morrison                           512.343.1010
Liant Software Corporation.   www.liant.com



Sun, 19 May 2002 03:00:00 GMT  
 Fujitsu COBOL 4.2 ISAM Questions
The capacity for a FJ ISAM (or relative or sequential file) is
1,073,741,824 bytes (more or less).

You might check with FJ for possible increased capacity in
their new version 5, due to be released shortly (more or less).

On another matter, my curiosity was piqued over what kind of
business would have 81 MILLION Social Security numbers.

From your web page:

The Merchants Association, via the TransUnion system, provides a
consumer credit and collection history, along with records of
addresses, employments, liens and judgments. The NATIONAL CREDIT
REPORT format can be tailored to meet customer's individual needs. The
MERGED REPORT combines an individual's or joint applicant's
TransUnion, TRW, and/or Equifax infile into one easy-to-read credit
report.

So now I know.


Quote:
> Hello All,

> We have a project coming up that will require loading approximately
81
> million 300 byte records into a file.  This file will then be
searched
> using a file from our mainframe as input.  The accesses necessary
will
> be 1. Name 2. SSN.

> Initial thought is to use ISAM since the indexes are straight
forward
> and I can build and search the file very easily directly from COBOL
> without any external database software involved - plus I have the
> search routines already written in COBOL on the mainframe.  The file
> is to be loaded, searched then removed.  This process would occur
> quarterly.  We are attempting to update addresses in our mainframe
> database with the more current addresses in this file.  Building the
> file on the mainframe is out - too costly to add the necessary
> disk/processor upgrade.  NT server would be much cheaper to put
> together and quicker!  After the match process the results would be
> used to update the mainframe database (estimate a 10% match rate).

> 1.  Can an ISAM file handle 81 million records?  Are there any
> limitations (other than the obvious like disk space) for ISAM files?

> 2.  I was told that Fujitsu ISAM files have the keys embedded in the
> main data file.  MF, I know, has a separate file for the indexes.
Is
> this true?

> 3.  I read through the Fujitsu documentation on the File Utility and
> it appears to me that I am going to need at least an equal amount of
> disk as the file I'm making into an ISAM - probably more like 2.5 -
3
> times, for the output and work files.  True?

> 4.  Assuming ISAM is out of the question, what would be your best
> guess/opinion for an alternative file/database format?  Keep in mind
> that I have VERY limited experience interfacing COBOL to anything
else
> (ODBC for example, minimal API and usually I get very {*filter*}y in the
> process :-0).

> Thanks for your opinions and assistance.

> Doug Blaze



Sun, 19 May 2002 03:00:00 GMT  
 Fujitsu COBOL 4.2 ISAM Questions
The keyword here is 'reliable'.  I don't know how reliable the Fujitsu
Indexed module is, but if this is an important application, I would
recommend Btrieve from Pervasive Software.  Actually, now they have a
newer product called SQL 2000.  The workstation engine (single user
version) only costs $49.  It is supposed to be faster than Btrieve,
and just this week I spooled off and recreated a Btrieve database of
about 3 million records, with *many* indexes (some files over 10), in
under 3 hours on a Dell PIII/550 with 256MB RAM.  The load time was 2
hours, 28 minutes.  And Btrieve is as solid as a rock.  That same
database has been running for 12 years, and we have never had to
reload it from data loss or corruption.  Against my advice, and much
to my dismay, the users have begun a habit of just turning off their
machines without exiting the application or Windows, sometimes in the
middle of a transaction.  Btrieve recovers with nary a blip.
--

Sun Valley Systems     http://www.*-*-*.com/
"For God so loved the world that He gave His only begotten Son, that
whoever believes in Him should not perish but have everlasting life."
Quote:

>We have a project coming up that will require loading approximately 81
>million 300 byte records into a file.  This file will then be searched
>using a file from our mainframe as input.  The accesses necessary will
>be 1. Name 2. SSN.

>Initial thought is to use ISAM since the indexes are straight forward
>and I can build and search the file very easily directly from COBOL
>without any external database software involved - plus I have the
>search routines already written in COBOL on the mainframe.  The file
>is to be loaded, searched then removed.  This process would occur
>quarterly.  We are attempting to update addresses in our mainframe
>database with the more current addresses in this file.  Building the
>file on the mainframe is out - too costly to add the necessary
>disk/processor upgrade.  NT server would be much cheaper to put
>together and quicker!  After the match process the results would be
>used to update the mainframe database (estimate a 10% match rate).

>1.  Can an ISAM file handle 81 million records?  Are there any
>limitations (other than the obvious like disk space) for ISAM files?

>2.  I was told that Fujitsu ISAM files have the keys embedded in the
>main data file.  MF, I know, has a separate file for the indexes.  Is
>this true?

>3.  I read through the Fujitsu documentation on the File Utility and
>it appears to me that I am going to need at least an equal amount of
>disk as the file I'm making into an ISAM - probably more like 2.5 - 3
>times, for the output and work files.  True?

>4.  Assuming ISAM is out of the question, what would be your best
>guess/opinion for an alternative file/database format?  Keep in mind
>that I have VERY limited experience interfacing COBOL to anything else
>(ODBC for example, minimal API and usually I get very {*filter*}y in the
>process :-0).

>Thanks for your opinions and assistance.

>Doug Blaze



Mon, 20 May 2002 03:00:00 GMT  
 Fujitsu COBOL 4.2 ISAM Questions

Quote:

> The keyword here is 'reliable'.  I don't know how reliable the Fujitsu
> Indexed module is, but if this is an important application, I would
> recommend Btrieve from Pervasive Software.  Actually, now they have a
> newer product called SQL 2000.  The workstation engine (single user
> version) only costs $49.  It is supposed to be faster than Btrieve,
> and just this week I spooled off and recreated a Btrieve database of
> about 3 million records, with *many* indexes (some files over 10), in
> under 3 hours on a Dell PIII/550 with 256MB RAM.  The load time was 2
> hours, 28 minutes.  And Btrieve is as solid as a rock.  That same
> database has been running for 12 years, and we have never had to
> reload it from data loss or corruption.  Against my advice, and much
> to my dismay, the users have begun a habit of just turning off their
> machines without exiting the application or Windows, sometimes in the
> middle of a transaction.  Btrieve recovers with nary a blip.

Judson,

Don't know Btrieve - but clarification - when you write 'created a
Btrieve database' is the word 'database' used loosely, covering both
flat-files and DB or do you mean it is specifically a DB ? Do you have a
preference for DB over flat-files, or do you use flat-files in your
'cheaper' applications where you also still use Screen Section.

Jimmy, Calgary AB



Mon, 20 May 2002 03:00:00 GMT  
 Fujitsu COBOL 4.2 ISAM Questions

Quote:


>> The keyword here is 'reliable'.  I don't know how reliable the Fujitsu
>> Indexed module is, but if this is an important application, I would
>> recommend Btrieve from Pervasive Software.  Actually, now they have a
>> newer product called SQL 2000.  The workstation engine (single user
>> version) only costs $49.  It is supposed to be faster than Btrieve,
>> and just this week I spooled off and recreated a Btrieve database of
>> about 3 million records, with *many* indexes (some files over 10), in
>> under 3 hours on a Dell PIII/550 with 256MB RAM.  The load time was 2
>> hours, 28 minutes.  And Btrieve is as solid as a rock.  That same
>> database has been running for 12 years, and we have never had to
>> reload it from data loss or corruption.  Against my advice, and much
>> to my dismay, the users have begun a habit of just turning off their
>> machines without exiting the application or Windows, sometimes in the
>> middle of a transaction.  Btrieve recovers with nary a blip.

>Judson,

>Don't know Btrieve - but clarification - when you write 'created a
>Btrieve database' is the word 'database' used loosely, covering both
>flat-files and DB or do you mean it is specifically a DB ? Do you have a
>preference for DB over flat-files, or do you use flat-files in your
>'cheaper' applications where you also still use Screen Section.

>Jimmy, Calgary AB

Sorry I wasn't clear.  Jimmy, the 'database' is a set of 18 Btrieve
indexed files, all but one with at least one key ('key' = an index on
one or more fields, field = 'key segment').  Most files, particularly
the large ones, have multiple keys (as many as 10), each with multiple
key segments (fields).  The operation I described was copying the
Btrieve files to flat files, then creating an empty set of Btrieve
files with expanded dates, then copying all the flat files to the
Btrieve files, expanding dates in the process.  Copying the data to
flat files took 0:12:10.  Creating the empty Btrieve files took less
than 1 second.  Copying the flat files (serially, one by one on one
machine) took 2:28:34.  The total number of records in all 18 files is
about 3 million.

I rarely use flat files, particularly in a transaction environment.
It is far too easy for them to get out of sync with the transactional
database files.  On transaction rollback, or reload and recovery from
a transaction log, the flat file would not be updated.  In the system
that uses this database, there is one file with no key and only one
record.  But I still made it a Btrieve file so Btrieve recovery would
insure its integrity.

In a few simple single user applications where database integrity is
not paramount, and the files can be reloaded easily, I have used COBOL
Indexed files.  But I probably wouldn't do that anymore, unless the
database was static or updated entirely by batch programs.  Pervasive's
SQL 2000 supports the old Btrieve interface, ODBC, SQL, and costs $49.
Any system I write is going to cost at least thousands.  Why skimp on
data integrity?  Anyway, both Micro Focus COBOL and Net Express can
use Btrieve for their Indexed file handler.  Almost any compiler can
use SQL 2000 through Btrieve API DLL calls, ODBC or SQL.  SQL 2000
supports both transactional and relational models, and the same data
store can be accessed using both modes.  Maximum file size is 64GB.

Pervasive's site is www.pervasive.com.  The SQL 2000 SDK is $149, the
Workstation Engine (single user version) is $49.  There is a Workgroup
version (for peer to peer and small server based networks).  A server
version is available for Linux, Solaris, NetWare and NT, but I don't
know the pricing.  You can download a free trial version of the SQL
2000 Workstation Engine from their website.
--

Sun Valley Systems    http://www.sunvaley.com
"For God so loved the world that He gave His only begotten Son, that
whoever believes in Him should not perish but have everlasting life."



Mon, 20 May 2002 03:00:00 GMT  
 Fujitsu COBOL 4.2 ISAM Questions

Quote:



> Sorry I wasn't clear.  Jimmy, the 'database' is a set of 18 Btrieve
> indexed files etc....

Judson, thanks for the detail. While flat-files have been OK with the
app over the past 20 years, if my technocrat does want to get into
'multi-user' then Btrieve/Pervasive is probably the way to go.
Thank you again.

Jimmy, Calgary AB



Mon, 20 May 2002 03:00:00 GMT  
 Fujitsu COBOL 4.2 ISAM Questions


[snip]

Quote:

> >> The keyword here is 'reliable'.  I don't know how reliable the Fujitsu
> >> Indexed module is, but if this is an important application, I would
> >> recommend Btrieve from Pervasive Software.

With due reverence for my friends up the street (Pervasive is in Austin as
well)...

Actually, the Fujitsu file system is quite reliable.

And so is that of RM/COBOL.

[large snip, testament to reliability]

Quote:
> Anyway, both Micro Focus COBOL and Net Express can
> use Btrieve for their Indexed file handler.

As do Fujitsu and RM/COBOL.  (I am guessing here but does AcuCOBOL supports
Btrieve as well?)

Quote:
> Almost any compiler can
> use SQL 2000 through Btrieve API DLL calls, ODBC or SQL.  SQL 2000
> supports both transactional and relational models, and the same data
> store can be accessed using both modes.

Fujitsu, RM/COBOL and Micro Focus COBOL files may also be accessed through
the relational model, without changing the physical layout of the file.
This is possible through the use of Relativity (Relativity also provides
ODBC access to Btrieve files).  See www.liant.com.

Quote:
> Maximum file size is 64GB.

I must admit that I am surprised that the limit is this low for a new
product -- but it should hold most folks for a while.

Tom Morrison
Liant Software Corporation



Mon, 20 May 2002 03:00:00 GMT  
 Fujitsu COBOL 4.2 ISAM Questions

Personally I like the tape solution, I think it would be fastest and
best for your purposes.  However here are the answers to your
questions.

You were asking specifically about Fujitsu COBOL.

Quote:
>1.  Can an ISAM file handle 81 million records?  Are there any
>limitations (other than the obvious like disk space) for ISAM files?

The standard Fujitsu File handler can handle: - this is from the
Fujitsu Read Me for v 4.2
H) Maximum file sizes

     In addition to the "normal version" of the COBOL file system
installed
     by default, the "large version" that allows you to use
larger-sized files
     is provided.

     The "normal version" is F3BIFRM.DLL (installed by default), and
the
     16-bit version is F1BCFRM.DLL. The maximum size of the file is as
     follows:

     - line sequential, record sequential, relative file: about 1G
       bytes.
     - indexed file: about 1.7G bytes.

     The "large version" is F3BIFRM.DLL (prepared on the CD-ROM),
which is
     not installed by the installer. The maximum sizes of files
     are as follows when using the "large version":

     - line sequential, record sequential, relative file: about 2G
       bytes.
     - indexed file: about 3.4G bytes.

     To use the "large version," please replace the installed
     F3BIFRM.DLL (32-bit "normal version") with the
     \COBOL32\COBOL97\LFILE\F3BIFRM.DLL ("large version") in CD-ROM.

     CAUTION: NEVER USE BOTH VERSIONS TOGETHER!

      - In this "large version," the method of giving the file
"exclusive"
        control is different from that in the "normal version."
Therefore,
        if "large version" and "normal version" access to the same
        file, exclusion/share processing of the file does not work
        correctly, and may cause file destruction.

        Operate as follows when you do not know if you are using the
large version
        or the normal version.

        - Execute the COBOL FILE UTILITY.
        - Select "Help".
        - Execute the command "About file utility".
        - If you used the large version, "FILE SIZE MODE: LARGE" is
          displayed in a dialog box.
          If you used the normal version, "FILE SIZE MODE: NORMAL" is
          displayed in a dialog box.

     NOTES

      - A COBOL file created by the "normal version" can be used as
        it is by the "large version".

      - The "large version" has been certified with the following
operating systems:
         -- Windows NT 4.0 (FAT / NTFS)
         -- Windows 95 (FAT / FAT32)
         -- Windows 98 (FAT / FAT32)

Quote:

>2.  I was told that Fujitsu ISAM files have the keys embedded in the
>main data file.  MF, I know, has a separate file for the indexes.  Is
>this true?

Fujitsu Embeds the keys, Micro Focus does not.  Fujitsu also
compresses the dataset by default.

Quote:
>3.  I read through the Fujitsu documentation on the File Utility and
>it appears to me that I am going to need at least an equal amount of
>disk as the file I'm making into an ISAM - probably more like 2.5 - 3
>times, for the output and work files.  True?  

I don't *THINK* so, solely because of the data compression Fujitsu
applies to indexed files.

Quote:
>4.  Assuming ISAM is out of the question, what would be your best
>guess/opinion for an alternative file/database format?  Keep in mind
>that I have VERY limited experience interfacing COBOL to anything else
>(ODBC for example, minimal API and usually I get very {*filter*}y in the
>process :-0).

An Oracle database could handle it, or as Judson suggested, Btrieve.
However I think the best solution is a tape to tape update.  That
would be fast, efficient and would conserve space.

---
Try a better search engine:   http://www.*-*-*.com/
My personal web site: http://www.*-*-*.com/



Tue, 21 May 2002 03:00:00 GMT  
 Fujitsu COBOL 4.2 ISAM Questions

Quote:
(Thane Hubbell) writes:

>An Oracle database could handle it, or as Judson suggested, Btrieve.
>However I think the best solution is a tape to tape update.  That
>would be fast, efficient and would conserve space.

How are you able to access Tape drives directly with Fujitsu?
Everything that I have looked at in the Professional Edition
seems to imply the requirement of files on disk or at least having
a domain name to attach to.  What products do you use for
tape interface?  How do you get the tape drives to be registered
with a device name or drive letter?


Tue, 21 May 2002 03:00:00 GMT  
 Fujitsu COBOL 4.2 ISAM Questions

Quote:

>(Thane Hubbell) writes:

>>An Oracle database could handle it, or as Judson suggested, Btrieve.
>>However I think the best solution is a tape to tape update.  That
>>would be fast, efficient and would conserve space.

>How are you able to access Tape drives directly with Fujitsu?
>Everything that I have looked at in the Professional Edition
>seems to imply the requirement of files on disk or at least having
>a domain name to attach to.  What products do you use for
>tape interface?  How do you get the tape drives to be registered
>with a device name or drive letter?

Sorry to mislead.  Even the 9 and 16 track tape drives available for
the PC are not accessible as files, they are simple file transfer
tools.

I was speaking of a sequential update on the mainframe.

---
Try a better search engine:  http://www.google.com
My personal web site: http://www.geocities.com/Eureka/2006/



Wed, 22 May 2002 03:00:00 GMT  
 
 [ 17 post ]  Go to page: [1] [2]

 Relevant Pages 

1. Fujitsu Software Corp special offering on Fujitsu COBOL 4.2 (V4L20)

2. Fujitsu Software Corp special offering on Fujitsu COBOL 4.2

3. Fujitsu COBOL 4.2

4. Fujitsu Cobol 4.2 with Win2000 pro

5. Re-Install Fujitsu COBOL 4.2

6. Fujitsu Cobol 4.2

7. fujitsu cobol 4.2

8. Fujitsu PowerCobol 4.2

9. fujitsu powercobol 4.2 mouse error

10. lock record problem on fujitsu v 4.2

11. Fujitsu 4.2 Runtime Installer

12. Nonexistent Fujitsu 4.2 upgrade / support

 

 
Powered by phpBB® Forum Software