CODEWITHSUNDEEP
javahadoopmapreduce
Aavik
Aavik

Reputation: 1037

Hadoop: Convert BytesWritable to byte[]

Could someone help me to convert BytesWritable to byte[]. I get additional bytes if I use value.getBytes() where value is BytesWritable.

Thanks

Upvotes: 1

Views: 1399

Answers (1)

Binary Nerd
Binary Nerd

Reputation: 13937

The problem you have is that BytesWritable contains a byte array and a length. You have to use these in combination. Here's a little unit test i wrote to show you whats happening:

@Test
public void testBytesWritable() {

    BytesWritable bw = new BytesWritable();
    Text t1 = new Text("ABCD");
    bw.set(t1.getBytes(), 0, t1.getLength());
    System.out.println("Size: " + bw.getBytes().length);

    Text t2 = new Text("A");
    bw.set(t2.getBytes(), 0, t2.getLength());
    System.out.println("Size: " + bw.getBytes().length);

    byte[] newArray = Arrays.copyOf(bw.getBytes(), bw.getLength());
    System.out.println("Size: " + newArray.length);
}

This prints out:

Size: 6
Size: 6
Size: 1

So you can see that the the BytesWritable is set twice, the first time with a longer set of bytes. Printing out the lengths of just the byte array shows that the length of the internal byte array doesnt reduce, it stays large. Thus you have to use the getLength() method to determine how many of the bytes in the BytesWritable are valid. In the unit test i've used Arrays.copyOf() to get the correct bytes, since it takes a byte array and a length.

Upvotes: 2

Related Questions