Skip to content

Instantly share code, notes, and snippets.

@nellshamrell

nellshamrell/redis_rust.md

Last active September 26, 2017 00:11
Show Gist options
  • Save nellshamrell/1c7a11d625cd4509946c9ca2cc35d1ed to your computer and use it in GitHub Desktop.
Save nellshamrell/1c7a11d625cd4509946c9ca2cc35d1ed to your computer and use it in GitHub Desktop.
Download ZIP
Raw
redis_rust.md

Intro

  • Title Slide
  • Who am I?
  • Why am I here?
  • Overview

Body

Habitat

  • Habitat 101

Rust

  • Rust 101
    • Rust is programming language that focuses on...
    • safety
    • speed
    • concurrency
    • performs majority of its safety checks and memory management decisions at compile time - so your program's runtime performance is not impacted
    • https://doc.rust-lang.org/book/second-edition/
    • small Rust example
  • brief intro to crates
  • When I started work on Habitat, Habitat used Redis as a persistant data store.

Redis and Rust

  • Intro to Redis crate used with Rust https://crates.io/crates/redis
    • rust implementation of a Redis client library
    • exposes general purpose interface to Redis
    • 2 api levels - low and high. high does not expose all the functionality of redis, might take liberties in how it speaks to the protocol. low lets you express any request on the redis level.
    • uses client object to open connection to Redis db (show example)
    • parameter to Client::open needs to implement the IntoConnectionInfo trait
    • can use cmd function to build redis requests - send query into connection, return value is the result object
  • Show prototype https://github.com/nellshamrell/redis-to-postgres-migrator
  • Reveal weirdness discovered with actual Habitat Redis store - protobuf!

Protobuf 101

  • Serialization
    • A way of serializing (or marshalling or packaging) data into a binary format in one place, than transmitted and unserialized in another place. In network programming we do this so that we can transmit data from one place on the network to another - the sender will use code to serialize the data, and the receiver will use that same code to unserialize the data.
    • Source: https://beej.us/guide/bgnet/output/html/singlepage/bgnet.html#serialization
  • Microservices
    • Serialization is especially useful because Habitat consists of several microservices.
  • Protobuf
    • from Google
    • platform for serializing structured data
    • define how you want your data to be structured once.
    • protobuf will generate code to easily serialize and unserialize your structured data, making it easy to transmit it over various streams and with various languages
    • How does protobuf work?
    • specify how you want the information you are serializing to be structured - define in a .proto file
    • Protocol buffer message - small logical record of information, containing series of name-value pairs
    • once you have defined message - you run a compiler for your application's language on your proto file. This generates code which, among other things, can be used to serialize the message into raw bytes and unserialize it.
    • Source: https://developers.google.com/protocol-buffers/docs/overview
  • How does protobuf work with redis?

Redis and Protobuf

  • show example

Converting Redis store to Postgres by time traveling through code

  • using protobuf serializations from 0.12.1 with code several major versions later.

The big data transfer

How Habitat uses Redis now

Conclusion

  • Takeaway
  • Who am I?
  • Questions?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment