n=d%*d% NO, bla=d% n=bla*bla YES, beginners question 
Author Message
 n=d%*d% NO, bla=d% n=bla*bla YES, beginners question

Hi all,
I tried to put my question into the subject line so that only people
who are interested would read this. I'm just starting with VB and I
like it. (I'm coming from fortran)

The code:
ndim% = Int(readndim.Text)
ntot = ndim% * ndim%

works only for ndim% up to 180. I tried to define ntot as lom (ntot&)
but I always get an overflow if I use ndim%=200

The code:
ndim% = Int(readndim.Text)
bla = ndim%
ntot = bla * bla

works and I don't understand why. I know I could be happy that it
works and ignore the extra line but this is a matter of interest. It
is probably trivial but trivial questions are never answered in
FAQ's...

Thanks for your time   Horst



Tue, 11 May 1999 03:00:00 GMT  
 n=d%*d% NO, bla=d% n=bla*bla YES, beginners question


: Hi all,
[snip]
: The code:
: ndim% = Int(readndim.Text)
: ntot = ndim% * ndim%

: works only for ndim% up to 180. I tried to define ntot as lom (ntot&)
: but I always get an overflow if I use ndim%=200

: The code:
: ndim% = Int(readndim.Text)
: bla = ndim%
: ntot = bla * bla

: works and I don't understand why. I know I could be happy that it
: works and ignore the extra line but this is a matter of interest. It
: is probably trivial but trivial questions are never answered in
: FAQ's...

        I believe that the conversion to type variant (bla = ndim%) is
what saves the second piece of code.  In effect, a% * b% probably attempts
to return something of type "%"; thus the overflow.  When you explicitly
perform (bla * bla), the value returned is probably of type Variant.  You
could simply convert one of the factors (ndim%) in your expression before
multiplying to avoid this error...*if* I'm right.

        -- Hira
<----------------------------------------------------------------------------->
  "shpe" = "he / she"; "shpis = her / his"; "shpim = her / him"

<----------------------------------------------------------------------------->



Tue, 11 May 1999 03:00:00 GMT  
 n=d%*d% NO, bla=d% n=bla*bla YES, beginners question



Quote:
> On Fri, 22 Nov 1996 05:34:04 GMT, Horst Stratemeier


Quote:
> : ndim% = Int(readndim.Text)
> : ntot = ndim% * ndim%
> : works only for ndim% up to 180.
>  I believe that the conversion to type variant (bla = ndim%) is
> what saves the second piece of code.  In effect, a% * b% probably attempts
> to return something of type "%"; thus the overflow.  When you explicitly
> perform (bla * bla), the value returned is probably of type Variant.  You
> could simply convert one of the factors (ndim%) in your expression before
> multiplying to avoid this error...*if* I'm right.

Hira,

Almost right !
When VB processes an expression, it uses the first parameter in each parse
to store the result of that parse. Hence it is trying to store 180*180
in ndim% (temporarily) before assigning it to ntot.

The way to get around it is,
   ntot = cLng(ndim%) * ndim%

Steve.



Thu, 13 May 1999 03:00:00 GMT  
 n=d%*d% NO, bla=d% n=bla*bla YES, beginners question

Quote:



> > On Fri, 22 Nov 1996 05:34:04 GMT, Horst Stratemeier

> > : ndim% = Int(readndim.Text)
> > : ntot = ndim% * ndim%

> > : works only for ndim% up to 180.

> >  I believe that the conversion to type variant (bla = ndim%) is
> > what saves the second piece of code.  In effect, a% * b% probably attempts
> > to return something of type "%"; thus the overflow.  When you explicitly
> > perform (bla * bla), the value returned is probably of type Variant.  You
> > could simply convert one of the factors (ndim%) in your expression before
> > multiplying to avoid this error...*if* I'm right.

> Hira,

> Almost right !
> When VB processes an expression, it uses the first parameter in each parse
> to store the result of that parse. Hence it is trying to store 180*180
> in ndim% (temporarily) before assigning it to ntot.

> The way to get around it is,
>    ntot = cLng(ndim%) * ndim%

> Steve.

Steve,
This is not true. If VB worked the way you say ndim% would be trashed
by the operation, and it's not. What happens is VB looks at the types
of the operands in an operation and takes the smallest that will fit
both. For example, with Integer * Integer VB performs an integer
multiplication. With Integer * Single VB would convert the Integer to
a single and then do a single presision operation. For more complex
expressions VB treats each seperatly. Thus:

d% = c!/ (a% * b%)

would first perform an integer multiplication of a% and b% yealding an
integer, which it would then convert to a single and do a single
precision division between c! and this result. The resultant single
preceision number would then be rounded and transfered to d%. In VB4
this goes a step further, for example:

a="123"
b%=10
debug.print a * b%

would produce 1230 when run within VB. The string is evaluated to a
number for the operation.

To answer the original question 200(Integer) * 200(Integer) = 40000
which is more that an integer can contain, thus the overflow.
You could stop it with:

nTot = clng(ndim%) * clng(ndim%)
However you could just change:

ndim% = Int(readndim.Text)

to

ndim& = clng(readndim.text)

which would produce a long integer in the first place.
HTH
Ken

----------------------------------------------------------------------

Applications Programmer/Network Manager  Phone : (+44) (0)1865 224149
Diabetes Research, RI, Oxford, England     Fax : (+44) (0)1865 723884



Fri, 14 May 1999 03:00:00 GMT  
 n=d%*d% NO, bla=d% n=bla*bla YES, beginners question

Quote:

> Hi all,
> I tried to put my question into the subject line so that only people
> who are interested would read this. I'm just starting with VB and I
> like it. (I'm coming from FORTRAN)

> The code:
> ndim% = Int(readndim.Text)
> ntot = ndim% * ndim%

> works only for ndim% up to 180. I tried to define ntot as lom (ntot&)
> but I always get an overflow if I use ndim%=200

> The code:
> ndim% = Int(readndim.Text)
> bla = ndim%
> ntot = bla * bla

> works and I don't understand why. I know I could be happy that it
> works and ignore the extra line but this is a matter of interest. It
> is probably trivial but trivial questions are never answered in
> FAQ's...

> Thanks for your time   Horst

Hello Horst;

In the 1st example you are multplying integers. Max 32,768+-.  I think I
have read that it generates an int temp to copy to your result.  180 x
180 = 32,400.  Any larger operands overflow the temp.

In the 2nd example the operands (bla) are varients.  A varient has the
range of double. So no overflow for the values you picked.

Look at help for data types.

/Don Ames



Sat, 05 Jun 1999 03:00:00 GMT  
 
 [ 5 post ] 

 Relevant Pages 

1. Multiple-step OLE DB operation bla bla bla...

2. bla bla test sheep

3. bla bla

4. Help, need API to get short filenames (such as: c:\progra~1\bla.txt)

5. Help, need API to get short filenames (such as: c:\progra~1\bla.txt)

6. Help, need API to get short filenames (such as: c:\progra~1\bla.txt)

7. Simple Yes/No question

8. Yes, another page numbering question

9. Batch file question (Yes, I know it's off-subject)

10. VB3 (yes, VB3!) database question

11. dbComboBox question -- (question from a beginner)

 

 
Powered by phpBB® Forum Software