CODEWITHSUNDEEP
unit-testinghadoop
zohar
zohar

Reputation: 2378

how to unit test Hadoop Writable

I Implemented WritableComparable of my own but I canoot find good wasy to unit test write and readFields methods.

Any ideas?

Upvotes: 2

Views: 1279

Answers (1)

Lorand Bendig
Lorand Bendig

Reputation: 10650

Probably you can find easier way to test you writables, but manually doing the serialization/deserialization will also work. E.g:

MyUtils.java:

...
import org.apache.commons.io.IOUtils;
...
public static byte[] serialize(Writable writable) throws IOException {
        ByteArrayOutputStream out = new ByteArrayOutputStream();
        DataOutputStream dataOut = null;
        try {
            dataOut = new DataOutputStream(out);
            writable.write(dataOut);
            return out.toByteArray();
        } 
        finally {
            IOUtils.closeQuietly(dataOut);
        }
    }

public static <T extends Writable> T asWritable(byte[] bytes, Class<T> clazz)
            throws IOException {
        T result = null;
        DataInputStream dataIn = null;
        try {
            result = clazz.newInstance();
            ByteArrayInputStream in = new ByteArrayInputStream(bytes);
            dataIn = new DataInputStream(in);
            result.readFields(dataIn);
        } catch (InstantiationException e) {
            // should not happen
            assert false;
        } catch (IllegalAccessException e) {
            // should not happen
            assert false;
        } finally {
            IOUtils.closeQuietly(dataIn);
        }
        return result;
    }

Then in your test class: 

CustomWritable record = ... ; //your initialized Writable
byte[] serializedBytes = MyUtils.serialize(record);

CustomWritable deserialized = 
  MyUtils.asWritable(serializedBytes, CustomWritable.class);

assertEquals("Value mismatch!", record.getFieldA(), deserialized.getFieldA());
...

Upvotes: 5

Related Questions