Using DEC Fortran to read raw binary data? 
Author Message
 Using DEC Fortran to read raw binary data?

Hi,

We are using DEC fortran under OpenVMS 6.2 on an DEC Alpha computer. We would
like to read in REAL and INTEGER values from a binary data file created on
another computer system.

Does anyone know of a way in DEC Fortran that we can open the binary file and
read the data byte by byte or in multi-byte blocks to transfer the data into
REAL and INTEGER variables in the Fortran program? We would need to be able to
read the binary data as a stream of bytes.

Opening the file as an unformatted binary file causes an error, which seems to
be related to Fortran assuming that some sort of record-related information
should be present (it isn't).

Would it be necessary to write some sort of file I/O routines in C and call them
from the Fortran program in order to accomplish this task? (I hope not.)

Any help with this problem is appreciated.

+-----------------------------+--------------------------------------------+
| Witold Waldman              | Aeronautical & Maritime Research Lab       |
| Senior Professional Officer | Airframes and Engines Division             |
| Tel: +61 3 9626 7389        | PO Box 4331, Melbourne VIC 3001, Australia |

+-----------------------------+--------------------------------------------+



Sun, 21 Jun 1998 03:00:00 GMT  
 Using DEC Fortran to read raw binary data?

Quote:

>We are using DEC Fortran under OpenVMS 6.2 on an DEC Alpha computer. We would
>like to read in REAL and INTEGER values from a binary data file created on
>another computer system.

This sort of question is asked so often it really ought to be in the FAQ,
but I don't think it is.  Briefly the answer is: you can't do it, at
least not easily, and not in all cases.  The best advice if you must read
a binary file as a stream is: don't use Fortran, or if you must use
Fortran don't use VMS ("Open" or not it makes no difference).

Often it is possible to access a binary file by opening it
  FORM='UNFORMATTED', ACCESS='DIRECT'
but this requires reading the file as a set of fixed-length records.
With unformatted records on VMS Fortran this is specified in 4-byte
units.  Doing this works fine on Unix systems and most others, but VMS
unfortunately stores the record-length in the file header and refuses to
open the file with any other length.

You may be able to examine the file header with DIR and find what VMS
thinks its length is, and use this (or not give it at all, VMS Fortran
allows an OPEN with no length specified, and uses what is in the file
header).  But your problems may still not be over.  If the file was
written on another system by a Fortran program which created it as an
unformatted file it will have record info attached to each record, which
will get in the way and have to be identified and ignored.  Also the byte
order or word formats of each real and integer may be different (but DEC
Fortran has OPEN options which can cope with some of these).
And if the file length is not an exact multiple of the record length then
the last incomplete record will not be accessible this way.  
--
-------------------------------------------------------------------------

Dept of Physics & Astronomy,        
University of Leicester.        



Mon, 22 Jun 1998 03:00:00 GMT  
 Using DEC Fortran to read raw binary data?


Quote:
(Witold Waldman) writes:

|>Hi,
|>
|>We are using DEC Fortran under OpenVMS 6.2 on an DEC Alpha computer. We would
|>like to read in REAL and INTEGER values from a binary data file created on
|>another computer system.
|>
|>Does anyone know of a way in DEC Fortran that we can open the binary file and
|>read the data byte by byte or in multi-byte blocks to transfer the data into
|>REAL and INTEGER variables in the Fortran program? We would need to be able to
|>read the binary data as a stream of bytes.
|>
|>Opening the file as an unformatted binary file causes an error, which seems to
|>be related to Fortran assuming that some sort of record-related information
|>should be present (it isn't).

I see that Clive Page has responded to your questions, but I do not agree with
most of his answers.  I'll respond to his posting separately.

The error you got opening the file as unformatted is likely caused by the
default assumption that a sequential, variable-length record file has
additional DEC Fortran "segment" information.  You can override this by
saying RECORDTYPE='VARIABLE' in the OPEN statement.

You don't say what the file looks like to VMS, how the data was written or
how it was transferred to the VMS system, so I can't give you as detailed
help as I'd like.

First, I will agree with Clive that Fortran doesn't provide the capability of
reading data files in arbitrary byte chunks - it is record oriented.  However,
that is not too difficult to work around (at least on VMS).

To help you further, I'd really need to know more about the file - the
output of a DIR/FULL command and a description of how it was created would
be helpful.  Please feel free to send me mail with the details.
--


DEC Fortran Development           WWW:  http://www.digital.com/info/slionel.html
Digital Equipment Corporation     CompuServe: 75263,3001
110 Spit Brook Road, ZKO2-3/N30
Nashua, NH 03062-2698             "Free advice is worth every cent"

For information on DEC Fortran 90, see http://www.digital.com/info/hpc/f90/



Mon, 22 Jun 1998 03:00:00 GMT  
 Using DEC Fortran to read raw binary data?

|>Often it is possible to access a binary file by opening it
|>  FORM='UNFORMATTED', ACCESS='DIRECT'
|>but this requires reading the file as a set of fixed-length records.
|>With unformatted records on VMS Fortran this is specified in 4-byte
|>units.  Doing this works fine on Unix systems and most others, but VMS
|>unfortunately stores the record-length in the file header and refuses to
|>open the file with any other length.

It is likely that the file in question has the variable-length record
attribute, therefore using ACCESS='DIRECT' won't work.    I'll also note
that we've added a /ASSUME=BYTERECL command switch to specify 1-byte units
for unformatted files.

In a separate response to Witold Waldman's posting, I said I disagreed with
most of your answers, but rereading your post, I see I misunderstood some of
it and you've given some good advice.  My apologies for the misunderstanding.
--


DEC Fortran Development           WWW:  http://www.digital.com/info/slionel.html
Digital Equipment Corporation     CompuServe: 75263,3001
110 Spit Brook Road, ZKO2-3/N30
Nashua, NH 03062-2698             "Free advice is worth every cent"

For information on DEC Fortran 90, see http://www.digital.com/info/hpc/f90/



Mon, 22 Jun 1998 03:00:00 GMT  
 Using DEC Fortran to read raw binary data?
DEC should be praised for its apparent decision to have Steve Lionel
to respond to many USENET questions involving DEC Fortran.

Not only is this a great improvement over the effort required to
obtain technical support from most compiler vendors, but it  
provides USENET users with an authoritative response.

It definitely strengthens DEC's Fortran product, and makes it much
more useful.

(BTW, I mostly use other brands of Fortran on other brands of machine.
But I can appreciate good technical support.)
-------------------------------------------------------------------------



Tue, 23 Jun 1998 03:00:00 GMT  
 Using DEC Fortran to read raw binary data?

Quote:

> DEC should be praised for its apparent decision to have Steve Lionel
> to respond to many USENET questions involving DEC Fortran.

I second the motion.

Every major software vendor should have qualified support persons
monitoring all relevant newsgroups to find out and respond to real user
concerns. In 1996, this should be considered a normal part of the
support operation.

If software group management thinks it is a waste of time, they should
be reminded that a newsgroup posting can answer MANY users at the same
time.

If somebody wants to know how to do it, you should watch Steve Lionel.

Good job, Steve.

--Loren Meissner



Tue, 23 Jun 1998 03:00:00 GMT  
 Using DEC Fortran to read raw binary data?


[snip]

Quote:
>_Steve Lionel's_ USENET monitoring should be praised, and DEC should

[snip]

Let me add a "me too" to that.  I do not currently use DEC equipment, but
I always check Steve's posts.  They are always well reasoned and well
written, and very often have good information of use to any FORTRAN user.

Ken Plotkin



Tue, 23 Jun 1998 03:00:00 GMT  
 Using DEC Fortran to read raw binary data?

<Blush!>  Wow!  What a great way to start the week (despite yet another
snowstorm outside...)

Thank you all for your kind words.  Though my management at Digital is
extremely supportive of my activities here, it's the compliments from users
which are truly important to me.  (I have, of course, forwarded copies of
your notes to my supervisor!)

I do want to note that I am just the most visible member of Digital's Fortran
development team (though the one who is primarily responsible for newsgroup
interaction) - many others read the group regularly and all recognize the
value in newsgroup participation.  When I joined the VAX FORTRAN team in
1978 (doing Run-Time Library work - I worked on various other products
and then joined the compiler team in 1988), I noted that they already had
the philosophy of doing "whatever it takes" to keep the customers satisfied,
even if that meant favoring informal channels.  That tradition continues
today.  In addition to being able to help people with problems, we often find
worthwhile suggestions for product improvements in postings.  Better
diagnostics, better support of other vendors' features, clearer documentation
and more have been prompted by things we've read here.

All of us in Digital Fortran engineering are dedicated to delivering the
highest quality compilers on the fastest systems.  Keeping in touch with
what Fortran users want and need is a large part of that, which is why I
participate here.  (Of course, there's also the small point that I genuinely
enjoy it!)

Thanks again.
--


DEC Fortran Development           WWW:  http://www.digital.com/info/slionel.html
Digital Equipment Corporation     CompuServe: 75263,3001
110 Spit Brook Road, ZKO2-3/N30
Nashua, NH 03062-2698             "Free advice is worth every cent"

For information on DEC Fortran 90, see http://www.digital.com/info/hpc/f90/



Fri, 26 Jun 1998 03:00:00 GMT  
 Using DEC Fortran to read raw binary data?

Quote:
(Kenneth Plotkin) writes:


> >_Steve Lionel's_ USENET monitoring should be praised, and DEC should

                                      ^^^^^^
The first time I saw a post in this thread I thought that someone had  
threatened Steve to stop it.  The post where he explains his employers  
support his posts to this newsgroup hasn't arrived at my home computer  
yet, but the message is clear: DEC understands the importance of this.

Quote:
> Let me add a "me too" to that.  I do not currently use DEC equipment,
> but I always check Steve's posts.  They are always well reasoned and
> well written, and very often have good information of use to any FORTRAN
> user.

Hah - let me tell you: I use DEC equipment daily - together with their  
fine f77 compiler (this is not even their top-of-the-line product).  This  
compiler (and run-time library) is a solid piece of software engineering:  
I yet have to find a fault with it (most other compilers break down under  
the 'heavy load' of weather forecasting, see URL below :-) - Either you  
get hit by a run-time library problem or one of the various optimisation  
passes produces garbage for which you have to find a work-around).  I'm  
glad to have access to another compiler that finds errors in _our_ code  
(like g77 does), instead of the other way around ...

Quote:
> Ken Plotkin

--

Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Phone: +31 346 214290; Fax: +31 346 214286
URL: http://www.knmi.nl/hirlam


Sun, 28 Jun 1998 03:00:00 GMT  
 Using DEC Fortran to read raw binary data?
I use the following under DEC Fortran to read binary files:

      OPEN(2,FILE='XXX',STATUS='OLD',
     &    FORM='UNFORMATTED',RECORDTYPE='STREAM')
      READ(2) I,J,K
      READ(2) X,Y,Z

Hope this works for you.

Warren Stephens
HNN Inc.



Sun, 28 Jun 1998 03:00:00 GMT  
 Using DEC Fortran to read raw binary data?

Quote:
>Hah - let me tell you: I use DEC equipment daily - together with their  
>fine f77 compiler (this is not even their top-of-the-line product).  This  
>compiler (and run-time library) is a solid piece of software engineering:  
>I yet have to find a fault with it (most other compilers break down under  
>the 'heavy load' of weather forecasting, see URL below :-) - Either you  
>get hit by a run-time library problem or one of the various optimisation  
>passes produces garbage for which you have to find a work-around).  I'm  
>glad to have access to another compiler that finds errors in _our_ code  
>(like g77 does), instead of the other way around ...
>> Ken Plotkin

I suspect you are right.  DEC provides fairly good quality stuff, even
if they seem to make a point of being different.  The place that DEC
operating systems and compilers are deficient in is their handling of
external data formats.

I have very often had to read files produced under VMS on other
platforms.  In only one case did the VMS programmer know the actual full
format that was produced--a person who had presumably produced lots of
CCTs (computer compatible tapes) for distribution to many agencies
by NOAA.  Ordinarally VMS simply hides the format, and their tape
DUMP utility sometimes doesn't even always get things right.  Further,
because of the record-oriented nature of their file system, if you
FTP a VMS file out to something else, and back again, it probably will
have lost information, even if you used BINARY.  This creates
nightmares for people trying to use VMS supplied data or convert VAX
programs for multi-platform use.  Just notice how many USENET questions
pertain to this problem.

DEC is not alone in this sort of problem.  That is why I have vainly
advocated the idea that Fortran needs to have a totally standardized
byte-stream data format, which can read or write ANY external data
format of all common media and TCP/IP network transmissions.  C/C++
mostly do have that capability, even though it isn't technically in the
standard (e.g., CHAR isn't always 8 bits, and raw data formats aren't
actually required), and you do have to work a bit at it.  Ideally
Fortran should go a step further and add conversion functions to convert
to and from all common integral and floating point formats now in use.

As I have said before, I believe that in today's multi-platform
environment, where you probably will throw away each machine in a
few years, much of the data is created by C/C++ programmers, and a
randomly chosen computer manufacturer will probably go bankrupt
anyway, something approaching true portability and true
ability to read and write ANYTHING are desperately needed.  

Standardized Fortran bit packing and (good) data compression would also
help in those areas (like remote sensing) where data volumes are huge.
(Those formats, like HDF, HTML, CEOS, et al, that are standardized,
are often too inefficient, in bits and time, to be practical.)

Is this such a big thing to ask?  If I thought it had the slightest
ghost of a chance, I would be happy to write up a simple spec to
begin with (I don't know enought about various media to take it
all the way), on my own time.  But I know it would be drowned by the
people working on 6 or 38 bit machines, by people working on operating
systems with serious inherent limiations, or by people who hate
standards...  Maybe true standards and full capabilities would make
Fortran more competitive.  I don't even bother much doing raw I/O
in Fortran anymore--C does the job.



Mon, 29 Jun 1998 03:00:00 GMT  
 
 [ 22 post ]  Go to page: [1] [2]

 Relevant Pages 

1. Reading SUN binary data files on a VMS DEC alpha

2. Fortran/C code to read LV binary data?

3. Using derived types in reading binary data...

4. Problems reading Fortran binary data files

5. help reading fortran generated binary data...

6. Reading a binary data set created by SAS into FORTRAN

7. Read the fortran unformatted binary data.

8. newbie:displaying raw binary data

9. Tkinter: passing raw binary data back and forth

10. Reading binary file, convert raw bytes

11. raw binary data and httplib

12. Reading a binary file / writing binary data to a file

 

 
Powered by phpBB® Forum Software