CODEWITHSUNDEEP
javaintbyte
Dmitry Frank
Dmitry Frank

Reputation: 10747

Java: correct way to cast "unsigned" byte to int

I have spent several hours on looking for bug in third-party implementation of stream cipher Rabbit. There are several bugs, but one of them:

/**
 * @param IV An array of 8 bytes
 */
public void setupIV(final byte[] IV) {
    short[] sIV = new short[IV.length>>1];
    for(int i=0;i<sIV.length;++i) {
        sIV[i] = (short)((IV[i << 1] << 8) | IV[(2 << 1) + 1]);
    }
    setupIV(sIV);
}

The problem here is that byte IV[i << 1] is casted to int, but since Java doesn't have unsigned types, any value >= 0x80 is casted wrong. Say, byte 0xff becomes 0xffffffff, not 0x000000ff how author of the code above expected.

So I have added simple function:

   private int byte2int(byte b){
      int ret = b;
      if (ret < 0){
         ret += 256;
      }
      return ret;
   }

It works, but I'm wondering if this is correct way to do what I need? This solution seems dumb somewhat.

Upvotes: 6

Views: 2046

Answers (2)

Suzan Cioc
Suzan Cioc

Reputation: 30097

Use library functions for this operation (in ByteBuffer class). You will be able to control endianness as a bonus. This will rely on efficiency of Java creators.

package tests.StackOverflow.q20776371;

import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.ShortBuffer;

public class q20776371 {


    public static void main(String[] args) {

        byte[] bb = { (byte)0xFF, (byte)0x01, (byte)0x02, (byte)0x03 };

        ByteBuffer buffer1 = ByteBuffer.wrap(bb);

        // set endianness
        // buffer1.order(ByteOrder.BIG_ENDIAN);
        // buffer1.order(ByteOrder.LITTLE_ENDIAN);

        ShortBuffer buffer2 = buffer1.asShortBuffer();

        short[] ss = new short[bb.length>>1];
        buffer2.get(ss);

        for(int i=0; i<ss.length; ++i) {
            System.out.println(String.format("%04x", ss[i]));
        }
    }


}

Upvotes: 0

Siddharth
Siddharth

Reputation: 1166

I'm not sure how much this will help, but you may convert a signed byte to its unsigned value by running a binary AND operation with 0xFF.

Taking this logic further, you could check for overflows after the cast by running a similar AND on the int, with the appropriate operand. This of course assumes that you're always expecting positive or zero valued numbers, or in other words unsigned numbers.

(short)(((IV[i << 1] & 0xFF) << 8) | (IV[(2 << 1) + 1] & 0xFF))

The above is the result of putting it all together, courtesy of Radiodef.

Upvotes: 7

Related Questions