Precision of 'Reall datatype 
Author Message
 Precision of 'Reall datatype

What is the number of digits that is being used in the real datatype?

I am writing a Java program that shall read Real-values from files generated
by an old Pascal-program, and have some problems finding out what the
precision of Pascal's 'Real' datatype is..

Knut Erik Ballestad



Wed, 18 Jun 1902 08:00:00 GMT  
 Precision of 'Reall datatype

On Wed, 24 Jun 1998 11:14:48 +0200, "Knut Erik Ballestad"

Quote:

>What is the number of digits that is being used in the real datatype?
>I am writing a Java program that shall read Real-values from files generated
>by an old Pascal-program, and have some problems finding out what the
>precision of Pascal's 'Real' datatype is..

The problem is much more complex than you might think. The number of
"digits" in terms of decimals is of no use to you if the reals are
stored in binary format, i.e. as a 6-byte-object. JAVA has no idea how
to interpret the TP Real format because it is non-standard. I suppose
that JAVA will only be able to handle IEEE float (4 bytes) and IEEE
double (8 bytes).

The only way to read the values and to convert them to floating point
numbers with the same value to be interpreted correctly by a compiler
using the IEEE formats is to read them in as a byte sequence and to
convert them "manually" to the IEEE double format.

I'm including a C function which would perform this translation, as I
don't know enough of JAVA. I hope that you may find your way
translating it:

Quote:
>-------------------------------------------------------------<

void TP_Real2double(unsigned short real[3], double *d)
/* converts TP REAL to IEEE double assuming
   16 bit unsigned short
*/
{
  unsigned short *dd = (unsigned short *)d;

  if ((real[0]&0xff)==0)
    d[0]=dd[1]=dd[2]=dd[3]=0;
  else {
    dd[3] = (real[2]&0x8000) |              /* sign */
            (((real[0]&0xff)+0x37e)<<4) |   /* exponent */
            ((real[2]&0x7fff)>>11);         /* mant 35..38 */
    dd[2] = (real[1]>>11)+(real[2]<<5);     /* mant 19..34 */
    dd[1] = (real[0]>>11)+(real[1]<<5);     /* mant 03..18 */
    dd[0] = (real[0]&0xff00)<<5;            /* mant 00..02 */
  }

Quote:
}
>-------------------------------------------------------------<

A more elegant solution would be to re-generate the Pascal data file
using DOUBLE instead of REAL - which is only possible if you have the
sources and the appropriate input to the program, of course.

Hope it helps
Horst



Wed, 18 Jun 1902 08:00:00 GMT  
 
 [ 3 post ] 

 Relevant Pages 

1. MSSQL Image datatype VS. Paradox Graphics DataType

2. MS SQL Server's datatypes

3. General SQL error on D4 + SQLServer6.5 + Datatype'text' + DBMemo

4. getting high precision arithmetic for BP7

5. Rounding Off or Precision Error

6. Currency Field lossing precision

7. Precision Timer

8. Arithmetic in multiple precision

9. Real precision

10. HELP !!! Table field precision

11. multiple precision arithmetic

12. Precision calculations

 

 
Powered by phpBB® Forum Software