Read Avro file from Pandas

avro_to_dataframe.py

import pandas
import fastavro

def avro_df(filepath, encoding):
    # Open file stream
    with open(filepath, encoding) as fp:
        # Configure Avro reader
        reader = fastavro.reader(fp)
        # Load records in memory
        records = [r for r in reader]
        # Populate pandas.DataFrame with records
        df = pandas.DataFrame.from_records(records)
        # Return created DataFrame
        return df

I had an error 'utf8' codec can't decode byte 0x83. This was solved by using open(filepath, 'rb'), where the b means to read the file in binary format. As already mentioned, this argument is the "mode", not "encoding".

It looks like you can save a line of code and avoid temporarily duplicating the data in memory by passing the reader iterable directly to from_records rather than loading it into a list first.