Copying byte sequence from integer/real to character array 
Author Message
 Copying byte sequence from integer/real to character array

Hi,

I am writing code to write a binary file. This binary file contains data
of different types non-sequentially so it is not very efficient to write
each element in turn.

I therefore want to create a large character array, and then copy the bit
pattern from different integer and real kinds to this character array, and
then simply write the entire character array to disk.

I think this might be possible using the transfer function: something
like,

mychar(1:8) = TRANSFER(myreal8,mold)

where myreal8 is REAL*8 and mychar is a character array. If this is
possible, what should I specify for mold for this and other data types? If
it is not possible, does anyone have any alternative suggestions for doing
this?

Thanks!

Andrew



Wed, 22 Aug 2007 21:37:51 GMT  
 Copying byte sequence from integer/real to character array


Here's our description of transfer, hoping that's sufficient.

Regards,

Mike Metcalf

The transfer function allows data of one type to be
transferred to another without the physical representation being altered.
This would be useful, for example, in
writing a data storage and retrieval system. The system
itself could be written for one type, default integer say, and other
types handled by transfers to and from that type, for example:

   integer          :: store
   character(len=4) :: word        ! To be stored and retrieved
   :
   store = transfer(word, store)   ! Before storage
   :
   word  = transfer(store, word)   ! After retrieval
   :

transfer (source, mold [,size]}
returns a result of type and type parameters those of mold.
When size is absent, the result
is scalar if mold is scalar, and it is of rank one and size
just sufficient to hold all of source if mold is array-valued.
When size is present, the result is of rank one and size size.
If the physical representation of the result is
as long as or longer than that of source, the result contains
source as its leading part and the value of the rest is
processor dependent;
otherwise the result is the leading part of source.
As the rank of the result can depend on whether or not size
is specified, the corresponding actual argument must not itself
be an optional dummy argument.



Thu, 23 Aug 2007 00:01:40 GMT  
 Copying byte sequence from integer/real to character array
...
Quote:
> I am writing code to write a binary file. This binary file contains data
> of different types non-sequentially so it is not very efficient to write
> each element in turn.

...

Michael provided reference to TRANSFER but I'm curious as to what you're
really after...

Is the data you want to write not available sequentially or are you
simply concerned about writing an intermix of various types of data or
something else?  It's not clear to me there's a benefit to be gained
here for sure...



Thu, 23 Aug 2007 00:26:56 GMT  
 Copying byte sequence from integer/real to character array

Quote:

> Hi,

> I am writing code to write a binary file. This binary file contains data
> of different types non-sequentially so it is not very efficient to write
> each element in turn.

> I therefore want to create a large character array, and then copy the bit
> pattern from different integer and real kinds to this character array, and
> then simply write the entire character array to disk.

> I think this might be possible using the transfer function: something
> like,

> mychar(1:8) = TRANSFER(myreal8,mold)

> where myreal8 is REAL*8 and mychar is a character array. If this is
> possible, what should I specify for mold for this and other data types? If
> it is not possible, does anyone have any alternative suggestions for doing
> this?

> Thanks!

> Andrew

If acceptable for your application, there are non-standard extensions in
some compilers that will allow you to write any datatype (any sequence)
out in its internal "binary" representation, without muss or fuss (or
transfer).  Which compiler/os?

--

Gary Scott

fortran Library:  http://www.fortranlib.com

Support the Original G95 Project:  http://www.g95.org
-OR-
Support the GNU GFortran Project:  http://gcc.gnu.org/fortran/index.html

Why are there two?  God only knows.

Democracy is two wolves and a sheep, voting on what to eat for dinner...
Liberty is a well armed sheep contesting the vote. - Thomas Jefferson



Thu, 23 Aug 2007 01:32:21 GMT  
 Copying byte sequence from integer/real to character array
Thanks for your replies. I have found out what I was really after (the
correct specification to use for mold). In the case of a real*8

mychar(1:8) = TRANSFER(myreal8,(/1_'00000000'/)

and for other data types the number of zeros is equal to the number of
bytes for that type.

This seems to work well.

I am doing this because of I have a large amount of data that I need to
write in (potentially) different formats in a repeating but non-sequential
(across types) format. This way I can just create a large matrix, and fill
by transfering other vectors and save it to disk using a single write
command.

Gary: I am using the Intel compiler on Win32. I would still be interested
to know the extensions you are refering to if available.



Thu, 23 Aug 2007 04:04:32 GMT  
 Copying byte sequence from integer/real to character array

Quote:
> Thanks for your replies. I have found out what I was really after (the
> correct specification to use for mold). In the case of a real*8
> mychar(1:8) = TRANSFER(myreal8,(/1_'00000000'/)
> and for other data types the number of zeros is equal to the number of
> bytes for that type.
> This seems to work well.

Man, if that works well I am sure Steve would appreciate a
bug report.  Your assignment statement creates an expression
with LEN = 8, KIND = 1, and SHAPE = (/1/) and assigns it to
a scalar.  If you want to go this route you can create named
constants to hold the size of everything:

integer, parameter :: dp = selected_real_kind(15,300)
integer, parameter :: ck = kind('x')
integer, parameter :: real8_size = size(transfer(1.0_dp,(/ck_'x'/)))
integer :: mychar_pos = 1

Then later you can create substrings at will:

mychar(mychar_pos:mychar_pos+real8_size-1) = &
   transfer(myreal8,repeat(ck_'x',real8_size))
mychar_pos = mychar_pos+real8_size

--
write(*,*) transfer((/17.392111325966148d0,6.5794487871554595D-85, &
6.0134700243160014d-154/),(/'x'/)); end



Thu, 23 Aug 2007 04:38:55 GMT  
 Copying byte sequence from integer/real to character array

Quote:
> Hi,

> I am writing code to write a binary file. This binary file contains data
> of different types non-sequentially so it is not very efficient to write
> each element in turn.

> I therefore want to create a large character array, and then copy the bit
> pattern from different integer and real kinds to this character array, and
> then simply write the entire character array to disk.

> I think this might be possible using the transfer function: something
> like,

> mychar(1:8) = TRANSFER(myreal8,mold)

> where myreal8 is REAL*8 and mychar is a character array. If this is
> possible, what should I specify for mold for this and other data types? If
> it is not possible, does anyone have any alternative suggestions for doing
> this?

> Thanks!

> Andrew

I guess I dont see the problem,  why dont you just do a binary write??
open (1,file='test.dat',form='binary')

write (1)  x, k, strings, etc
and  continue adding to file later
write (1) more, data, etc

later you can read the file substituting a binary read  (without format
specifiers) recovering the data,

For amu{*filter*}t, I wrote a routine that allows creating a big string of any
data desired that
can be added to as needed  (anything to avoid using TRANSFER  I say) and
then output it
formatted.

! ---------------------
program output_formatted_string   ! with various type variables
real(8) :: x(10) = 123456789_8
integer :: k(10) = 123456789
character(10) :: astring = '1234567890'
character(2000) :: big_string

!  JUST DO IT DIRECTLY !!
write (*,99) x, k, astring        ! outputs 130 chars + cr lf

! BUT if you insist on creating a big blob of char data in memory first

big_string = v2s(loc(x),sizeof(x)) //  v2s(loc(k),sizeof(k))

! and later add to it

big_string = trim(big_string) //  astring       ! etc.
write (*,99) trim(big_string)   ! outputs same 130 chars + cr lf   as above

99 format (999999a)
stop

contains
function v2s(vloc,vsize)    ! reals, integers, types, scalars, arrays, you
name it  -> string
integer   :: vloc,vsize
character(vsize) :: string  ; pointer (vloc,string)
character(vsize) :: v2s
v2s = string
end function
end program



Thu, 23 Aug 2007 17:35:17 GMT  
 Copying byte sequence from integer/real to character array


Quote:
>> mychar(1:8) = TRANSFER(myreal8,(/1_'00000000'/)

>> and for other data types the number of zeros is equal to the number of
>> bytes for that type.

>> This seems to work well.

>Man, if that works well I am sure Steve would appreciate a
>bug report.  Your assignment statement creates an expression
>with LEN = 8, KIND = 1, and SHAPE = (/1/) and assigns it to
>a scalar.

I got the bug report - what it actually does in this case is use a mold of
(/'00000000'/) - the 1_ is ignored.  A bizarre parsing bug which we will of
course fix.

There is also, as you infer, a shape mismatch here, which our compiler does
diagnose, at least in the cases I try.

Steve Lionel
Software Products Division
Intel Corporation
Nashua, NH

User communities for Intel Software Development Products
  http://softwareforums.intel.com/
Intel Fortran Support
  http://developer.intel.com/software/products/support/



Fri, 24 Aug 2007 23:27:24 GMT  
 Copying byte sequence from integer/real to character array
On Mon, 07 Mar 2005 10:27:24 -0500, Steve Lionel

Quote:

>There is also, as you infer, a shape mismatch here, which our compiler does
>diagnose, at least in the cases I try.

My error - no shape mismatch.  The target is an array of single characters.

Steve Lionel
Software Products Division
Intel Corporation
Nashua, NH

User communities for Intel Software Development Products
  http://softwareforums.intel.com/
Intel Fortran Support
  http://developer.intel.com/software/products/support/



Fri, 24 Aug 2007 23:31:23 GMT  
 Copying byte sequence from integer/real to character array

Quote:
> I got the bug report - what it actually does in this case is use a mold of
> (/'00000000'/) - the 1_ is ignored.  A bizarre parsing bug which we will
of
> course fix.

How can you tell the 1_ is ignored?  Isn't the KIND parameter of
default CHARACTER 1 in Intel's as in most vendors' compilers?

Quote:
> There is also, as you infer, a shape mismatch here, which our compiler
does
> diagnose, at least in the cases I try.

This was the part I was worried about.  I couldn't think of any
reasonable extension that would turn this shape mismatch into
a desirable result.  But if it's diagnosed, how could it have
worked well?

--
write(*,*) transfer((/17.392111325966148d0,6.5794487871554595D-85, &
6.0134700243160014d-154/),(/'x'/)); end



Fri, 24 Aug 2007 23:54:23 GMT  
 Copying byte sequence from integer/real to character array

Quote:
> On Mon, 07 Mar 2005 10:27:24 -0500, Steve Lionel

> >There is also, as you infer, a shape mismatch here, which our compiler
does
> >diagnose, at least in the cases I try.
> My error - no shape mismatch.  The target is an array of single

characters.

OK, then it's a LEN mismatch.  Perhaps the O.P. didn't chack
his results carefully before concluding that it worked.  It
should just produce nonsense in my view, with no possibility
of reconstructing the original floating point numbers.

--
write(*,*) transfer((/17.392111325966148d0,6.5794487871554595D-85, &
6.0134700243160014d-154/),(/'x'/)); end



Sat, 25 Aug 2007 00:02:15 GMT  
 Copying byte sequence from integer/real to character array


...

Quote:
> >> mychar(1:8) = TRANSFER(myreal8,(/1_'00000000'/)
...
> I got the bug report - what it actually does in this case is use a mold of
> (/'00000000'/) - the 1_ is ignored.  A bizarre parsing bug which we will of
> course fix.

What's the bug in that? I would think that it *SHOULD* mean the same
thing with and without the 1_. I assume that the kind number for default
character is 1.

You might have forgotten about the somewhat strange (IMO) form of kind
specifiers for character literals.  Namely, the kind specifier precedes
instead of follows the main part of the literal. You might be unfamiliar
with the form because it is quite rare to see people using kind
constants with character literals (largely because it is at least
somewhat rare for current compilers to support more than one kind of
character). Or then, perhaps I'm the one confused by not seeing this
very often.

Quote:
> There is also, as you infer, a shape mismatch here, which our compiler does
> diagnose, at least in the cases I try.

I presume this was the issue that it was suggested would be worth a bug
report (if it had gone through without warning).

--
Richard Maine                       |  Good judgment comes from experience;
email: my first.last at org.domain  |  experience comes from bad judgment.
org: nasa, domain: gov              |        -- Mark Twain



Sat, 25 Aug 2007 00:14:48 GMT  
 Copying byte sequence from integer/real to character array


Quote:
>You might have forgotten about the somewhat strange (IMO) form of kind
>specifiers for character literals.  Namely, the kind specifier precedes
>instead of follows the main part of the literal. You might be unfamiliar
>with the form because it is quite rare to see people using kind
>constants with character literals (largely because it is at least
>somewhat rare for current compilers to support more than one kind of
>character). Or then, perhaps I'm the one confused by not seeing this
>very often.

Sigh - yes.  I had indeed forgotten about that.  So, no bug there.  And you
and James are right that this code doesn't do what Andrew wants.

Steve Lionel
Software Products Division
Intel Corporation
Nashua, NH

User communities for Intel Software Development Products
  http://softwareforums.intel.com/
Intel Fortran Support
  http://developer.intel.com/software/products/support/



Sat, 25 Aug 2007 04:07:10 GMT  
 Copying byte sequence from integer/real to character array

Quote:
>this code doesn't do what Andrew wants

Whether the shapes conform depend on how mychar is defined.

If you have,

character(8) :: mychar

then the shapes do not conform and you get a compiler error.

BUT if you have

character(1) :: mychar(8)

they do and this is how they have been used in my code (except I am
dealing with arrays rather than scalars). As far as I am aware this works
and does what I want.



Sat, 25 Aug 2007 05:11:37 GMT  
 
 [ 14 post ] 

 Relevant Pages 

1. Using Python arrays of bytes for ILU SEQUENCE OF BYTE -- API is inadequate

2. Long integer <- -> sequence of bytes

3. Reading a sequence of (characters|bytes) from a file in Common Lisp

4. copy-sequence() and <array>

5. (INTEGER/REAL) to (CHARACTER) conversion

6. Changing character to real or integer

7. INTEGER, CHARACTER or REAL?

8. Question about converting integer or real number to character

9. Integer to an array of byte

10. Converting INTEGER to a BYTE array, LISP style

11. convert bytes of numerics to character arrays

12. swapping bytes on a two byte integer

 

 
Powered by phpBB® Forum Software