CODEWITHSUNDEEP
pythonavro
OmiO
OmiO

Reputation: 43

Iterate on avro DatumReader in Python

I am trying to write an avro consumer where data is byte encoded and sent without the schema. I followed https://stackoverflow.com/a/25130722/2276564. How can I iterate over the datumreader, I have no way of knowing how many records will come through the bytestream and would like to iterate over or read all records all at once. I suppose I cannot use the DataFileReader as the schema is stripped away and the stream is encoded.

I am currently looping through datumreader and using error handling to manage the situation where it will reach end of stream.

Any Clean/better way to achieve this?

Upvotes: 1

Views: 1087

Answers (1)

OmiO
OmiO

Reputation: 43

I solved the problem by using .tell() method of io.BytesIO which returns the current position of the buffer. After each read the program queries current position and check if its less than the length of the payload given to the buffer.

 while bytes_reader.tell() < len(payload):
        records.append(reader.read(decoder))

Upvotes: 2

Related Questions