CODEWITHSUNDEEP
javadatetimeserializationavro
Miguel L.
Miguel L.

Reputation: 728

How to serialize a Date using AVRO in Java

I'm actually trying to serialize objects containing dates with Avro, and the deserialized date doesn't match the expected value (tested with avro 1.7.2 and 1.7.1). Here's the class I'm serializing :

import java.text.SimpleDateFormat;
import java.util.Date;

public class Dummy {
    private Date date;
    private SimpleDateFormat df = new SimpleDateFormat("dd/MM/yyyy hh:mm:ss.SSS");

    public Dummy() {
    }

    public void setDate(Date date) {
        this.date = date;
    }

    public Date getDate() {
        return date;
    }

    @Override
    public String toString() {
        return df.format(date);
    }
}

The code used to serialize / deserialize : 

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.util.Date;

import org.apache.avro.Schema;
import org.apache.avro.io.DatumReader;
import org.apache.avro.io.DatumWriter;
import org.apache.avro.io.Decoder;
import org.apache.avro.io.DecoderFactory;
import org.apache.avro.io.Encoder;
import org.apache.avro.io.EncoderFactory;
import org.apache.avro.reflect.ReflectData;
import org.apache.avro.reflect.ReflectDatumReader;
import org.apache.avro.reflect.ReflectDatumWriter;

public class AvroSerialization {

    public static void main(String[] args) {
        Dummy expected = new Dummy();
        expected.setDate(new Date());
        System.out.println("EXPECTED: " + expected);
        Schema schema = ReflectData.get().getSchema(Dummy.class);
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        Encoder encoder = EncoderFactory.get().binaryEncoder(baos, null);
        DatumWriter<Dummy> writer = new ReflectDatumWriter<Dummy>(schema);
        try {
            writer.write(expected, encoder);
            encoder.flush();
            Decoder decoder = DecoderFactory.get().binaryDecoder(baos.toByteArray(), null);
            DatumReader<Dummy> reader = new ReflectDatumReader<Dummy>(schema);
            Dummy actual = reader.read(null, decoder);
            System.out.println("ACTUAL: " + actual);
        } catch (IOException e) {
            System.err.println("IOException: " + e.getMessage());
        }
    }
}

And the output :

EXPECTED: 06/11/2012 05:43:29.188
ACTUAL: 06/11/2012 05:43:29.387

Is it related to a known bug, or is it related to the way I'm serializing the object ?

Upvotes: 18

Views: 60973

Answers (3)

teu
teu

Reputation: 979

Avro 1.8 now has a date "logicalType", which annotates int. For example:

{"name": "date", "type": "int", "logicalType": "date"}

Quoting the spec:

A date logical type annotates an Avro int, where the int stores the number of days from the unix epoch, 1 January 1970 (ISO calendar).

Upvotes: 33

Raghubir Singh
Raghubir Singh

Reputation: 227

Issue Resolved at my end. I was also facing the same issue with serialisation of date field. Though Avro takes Date type as int. However once you declare it in this way

{ "name": "DateEffective", "type": [ "null", { "type": "int", "logicalType": "date" } ],

and run avrogen -s command than it makes C#/Java model class with date field.

Upvotes: 1

amas
amas

Reputation: 604

I think AVRO doesn't serialize date at this point. What I would do is to wrap it in another class and store at as a long (date.gettime()) while avro folks add this feature. And the reason that you see different Date values is that every time that you (and avro) create a Date object, it initializes the Date with the current System time.

Upvotes: 7

Related Questions