CODEWITHSUNDEEP
avro
Joha
Joha

Reputation: 965

How to show AVRO data from one field in two columns?

Let's say I have an AVRO file that was written as a record with one column named c1. When I read this file with the schema it was written with I get all the data from c1.

Now, is it possible to read exactly that file with another schema, that has for example three columns? One column named c0 with a default value of null, one column named c1 that returns all the values as with the first schema. And one column named c2 with an alias of c1 that also returns all values of c1?

Upvotes: 0

Views: 516

Answers (1)

Matthieu
Matthieu

Reputation: 4721

This is probably implementation dependent. If using JavaScript is an option, avsc will let you do what you want. For example, if you write an Avro file using your first schema...

const writerSchema = {
  type: 'record',
  name: 'Foo',
  fields: [{name: 'c1', type: 'int'}]
};

const encoder = avro.createFileEncoder('data.avro', writerSchema);

// Write a little data.
encoder.write({c1: 123});
encoder.write({c1: 45});
encoder.end({c1: 6789});

...then you can read it with the second schema simply as...

const readerSchema = {
  type: 'record',
  name: 'Foo',
  fields: [
    {name: 'c0', type: ['null', 'int'], 'default': null},
    {name: 'c1', type: 'int'},
    {name: 'c2', aliases: ['c1'], type: 'int'},
  ]
};

// Decode the file and print out its data.
avro.createFileDecoder('data.avro', {readerSchema})
  .on('data', (val) => { console.log(val); });

...and the output will have the form you expect:

Foo { c0: null, c1: 123, c2: 123 }
Foo { c0: null, c1: 45, c2: 45 }
Foo { c0: null, c1: 6789, c2: 6789 }

Upvotes: 1

Related Questions