Skip to content

Instantly share code, notes, and snippets.

@ivsanro1
Created March 31, 2022 10:12
Show Gist options
  • Select an option

  • Save ivsanro1/7b12a4a350832a8f86c9e9d325c1d5d7 to your computer and use it in GitHub Desktop.

Select an option

Save ivsanro1/7b12a4a350832a8f86c9e9d325c1d5d7 to your computer and use it in GitHub Desktop.
Easily read avro records from bytes
def read_avro(bytes_buff_or_file):
import avro.schema
from avro.datafile import DataFileReader, DataFileWriter
from avro.io import DatumReader, DatumWriter
from io import BytesIO
import json
if type(bytes_buff_or_file) == bytes:
from tempfile import TemporaryFile
# Has to be a file yes or yes because of how avro lib is made
with TemporaryFile() as tmp:
tmp.write(bytes_buff_or_file)
reader = DataFileReader(tmp, DatumReader())
return [json.loads(o.decode()) for o in reader]
else:
raise NotImplementedError
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment