ATT: Math wizards. Bit manipulations and integer overflow 
Author Message
 ATT: Math wizards. Bit manipulations and integer overflow

Here's one area where Basic as a language vs.. C is killing me:
I've got a number n from 0 to x.  I want to return the word and bitset in an
integer array that corresponds to that number.  For example if n = 27, I
want to know a.) word 1 and b.) bit 11 = 2048.  The code I use is:

word = n \ 16
bit = 2 ^ (n Mod 16)

There's no problem with the word value but the bit value gives me a problem
if it ends up being the high bit of a word.  In this case I get 2 ^ (15)
which always ends up being 32768.  I can't assign this to an integer because
it results in an overflow.  I have to assign the result to a property in an
OCX I didn't code so I can't use a long value.  I don't like the kludge of
If(bit = 32768) then bit = -32768 else ....   What code can I use to take a
number from 0 to 15 and return the resulting integer value?

How I long for bit shifting, unsigned integers ....



Mon, 03 Jul 2000 03:00:00 GMT  
 ATT: Math wizards. Bit manipulations and integer overflow


Fri, 19 Jun 1992 00:00:00 GMT  
 ATT: Math wizards. Bit manipulations and integer overflow

Hi Scott,

Well, you have to either, "kludge in VB code" (I do this for some API calls
that expect unsigned Integers), or use a DLL like Stamina, which does all of
the heavy lifting.  It is possible to write almost anything that you want in
pure VB code -- but this results in what I assume you don't want.  The extra
work, and poor performance, of pure VB code leads me to suggest the add-on
DLL approach.

If you are interested, you can get Stamina at a discount by clicking on the
VBxpert link on my homepage.  (This doesn't cut Jim Mack completely out of
the loop. <g>)

--
Richard Grier
Hard & Software
12962 West Louisiana Avenue
Lakewood, CO  80228
303-986-2179 (voice)
303-986-3143 (fax)
Author of Visual Basic Programmer's Guide to Serial Communications.
For information, look on my homepage at
http://ourworld.compuserve.com/homepages/richard_grier.
Use the Books link to order from Amazon.com.  For faster service, contact
the publisher at http://www.mabry.com.



Mon, 03 Jul 2000 03:00:00 GMT  
 ATT: Math wizards. Bit manipulations and integer overflow

Scott -
     For a couple of dozen bit manipulation functions, take a look at my
company's Stamina product.  It contains 300+ routines written in ASM,
including DxBit, DxSetBit, DxClearBit, DxParity and many others.  There's
even a complete API for dealing with Bit arrays.

--
     Jim Mack
     MicroDexterity, Inc
     PO Box 5372
     Plymouth, MI  48170-5372

     http://www.microdexterity.com



     Fax:  +1 734-453-8942

Quote:

> Here's one area where Basic as a language vs.. C is killing me:
> I've got a number n from 0 to x.  I want to return the word and bitset in
an
> integer array that corresponds to that number.  For example if n = 27, I
> want to know a.) word 1 and b.) bit 11 = 2048.  The code I use is:

> word = n \ 16
> bit = 2 ^ (n Mod 16)

> There's no problem with the word value but the bit value gives me a problem
> if it ends up being the high bit of a word.  In this case I get 2 ^ (15)
> which always ends up being 32768.  I can't assign this to an integer
because
> it results in an overflow.  I have to assign the result to a property in an
> OCX I didn't code so I can't use a long value.  I don't like the kludge of
> If(bit = 32768) then bit = -32768 else ....   What code can I use to take a
> number from 0 to 15 and return the resulting integer value?

> How I long for bit shifting, unsigned integers ....



Tue, 04 Jul 2000 03:00:00 GMT  
 ATT: Math wizards. Bit manipulations and integer overflow

Thank you, Mr. Mack.   :-)   nice try.

And now to answer Scott's question...  You can deal with bit values in VB.
It can sometimes be awkward, although mostly the only thing I miss are
shift and rot operations.  Okay.  The first step is to throw out Integers
for the moment and cast everything into Long Integers.

Try this:
        - Place 2 TextBoxes on a new Project's default Form1.
        - Place a CommandButton on the form.
        - In the form's code module place the following code:
        - Run the program.
        - Type 27 (or 49572 or whatever) into Text1.
        - Click Command1.

Private Type LongA
    Low As Integer
    High As Integer
End Type

Private Type LongB
    Value As Long
End Type

Private Sub Command1_Click()
    Dim word As Integer, bit As Integer
    Dim lval As Long    
    Dim a As LongA, b As LongB

    lval = CLng(Val(Text1))
    word = CInt(lval \ 16&)
    b.Value = 2& ^ (lval Mod 16&) ' Grab the Long bit value

    LSet a = b
    bit = a.Low ' grab just the low word.

    Text2 = "Word = " & word & vbCrLf & "Bit = " & bit
End Sub

By using LSet and two UDT's, you can easily split off the low WORD of a
long and keep it.  I do not pretend to be a bit-manipulation guru... there
are probably a million ways to do this in "pure" VB.

------------
Jim Houghtaling

email:  hmm... no.
web:   working on it (heh)
platform(s):  Win95 os/r2, WinNT 4.0 sp3, VB5ee sp3
('till I get my page up go to VBNet...It's the best I've seen yet.)
http://home.sprynet.com/sprynet/rasanen/vbnet/default.htm
------------
It often shows an excellent command
of language to say nothing.



Quote:
> Here's one area where Basic as a language vs.. C is killing me:
> I've got a number n from 0 to x.  I want to return the word and bitset in
an
> integer array that corresponds to that number.  For example if n = 27, I
> want to know a.) word 1 and b.) bit 11 = 2048.  The code I use is:

> word = n \ 16
> bit = 2 ^ (n Mod 16)

> There's no problem with the word value but the bit value gives me a
problem
> if it ends up being the high bit of a word.  In this case I get 2 ^ (15)
> which always ends up being 32768.  I can't assign this to an integer
because
> it results in an overflow.  I have to assign the result to a property in
an
> OCX I didn't code so I can't use a long value.  I don't like the kludge
of
> If(bit = 32768) then bit = -32768 else ....   What code can I use to take
a
> number from 0 to 15 and return the resulting integer value?



Tue, 04 Jul 2000 03:00:00 GMT  
 ATT: Math wizards. Bit manipulations and integer overflow

Thanks to all for your feedback.  I checked out the product mentioned and it
is impressive.  At this point all I'm looking at is the (bit 0 - 15) to
(signed integer) conversion.  Jim,  I used your snippet inline and it works
as advertised.  I guess I'll have to put together a little profiling program
to see how efficient it is compared to checking for 32768 and changing the
sign.  I just can't understand why doing assigning 32768 AND -1 (bitwise
AND) to an integer still results in an overflow.  Isn't the compiler smart
enough to know what's happening? Its refreshing to see you guys tackle these
issues through this group.  Thanks again.



Tue, 04 Jul 2000 03:00:00 GMT  
 ATT: Math wizards. Bit manipulations and integer overflow

Scott -
    You can't assign 32768 to a signed integer, (upper limit 32767), and
since the result of the expression (32768 And -1) is 32768, you're in the
same boat.

   This suggests a simple test for bit 15 of an integer:

     Bit15 = (Variable% < 0)

   If you need to determine which bits are set in an integer, and you can't
assign to a long integer, then you're stuck with some sort of special case,
or with splitting the integer into bytes (which you have seen how to do).

   But a long integer intermediate makes it very easy:

   TestLong& = Variable%

   Now you can test bit 15 just like all the others...

     Bit15 = (TestLong And 32768&)

        -- Jim

Quote:

> Thanks to all for your feedback.  I checked out the product mentioned and
it
> is impressive.  At this point all I'm looking at is the (bit 0 - 15) to
> (signed integer) conversion.  Jim,  I used your snippet inline and it works
> as advertised.  I guess I'll have to put together a little profiling
program
> to see how efficient it is compared to checking for 32768 and changing the
> sign.  I just can't understand why doing assigning 32768 AND -1 (bitwise
> AND) to an integer still results in an overflow.  Isn't the compiler smart
> enough to know what's happening? Its refreshing to see you guys tackle
these
> issues through this group.  Thanks again.



Tue, 04 Jul 2000 03:00:00 GMT  
 
 [ 7 post ] 

 Relevant Pages 

1. Long = Integer * Integer Overflow error

2. Bit manipulation and bit arrays

3. VB5 Math Overflow Error (fwd)

4. VB5 Math Overflow Error

5. Overflow on simple math?

6. Large integer math in VB6?

7. Is there a limit on a integer variable (overflow error)

8. Bitwise manipulation of integer

9. Overflow exceptions when converting from Long to Integer?

10. Integer overflow traps

11. unsigned 32 bits integer arithmetic / overflow

12. HELP: Integer -> Hex conversion (Overflow)

 

 
Powered by phpBB® Forum Software