Skip to content

Instantly share code, notes, and snippets.

@amir20
Forked from jeffkole/gist:4047435
Created November 9, 2012 21:51
Show Gist options
  • Save amir20/4048488 to your computer and use it in GitHub Desktop.
Save amir20/4048488 to your computer and use it in GitHub Desktop.
Avro IO with different reader and writer schemas
WRITER_SCHEMA = <<-JSON
{ "type": "record",
"name": "User",
"fields" : [
{"name": "username", "type": "string"},
{"name": "age", "type": "int"},
{"name": "verified", "type": "boolean", "default": "false"}
]}
JSON
READER_SCHEMA = <<-JSON
{ "type": "record",
"name": "User",
"fields" : [
{"name": "username", "type": "string"},
{"name": "verified", "type": "boolean", "default": "false"}
]}
JSON
# write a file
file = File.open('raw.avro', 'w')
encoder = Avro::IO::BinaryEncoder.new(file)
writer_schema = Avro::Schema.parse(WRITER_SCHEMA)
writer = Avro::IO::DatumWriter.new(writer_schema)
writer.write({"username" => "tom", "age" => 25, "verified" => true }, encoder)
writer.write({"username" => "dick", "age" => 23, "verified" => false }, encoder)
writer.write({"username" => "harry", "age" => 34, "verified" => false }, encoder)
file.flush
file.close
# read the file
file = File.open('raw.avro', 'r')
decoder = Avro::IO::BinaryDecoder.new(file)
reader_schema = Avro::Schema.parse(READER_SCHEMA)
reader = Avro::IO::DatumReader.new(writer_schema, reader_schema)
p reader.read(decoder)
p reader.read(decoder)
p reader.read(decoder)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment