Redis

redis-1-intro.md

Type of DBS

RDBMs

• Support queries / indexing / ...

DOC DB

• Data organized in collections of documents, Support queries / indexing / ...

Key-Value store

• Data organized by keys, no query or indexing.

Redis

-> Redis is an in-memory data structure store (!= DB, != Cache)

Have nots

• indexing
• query langage
• organization of the data in sub-collections (= just a list of keys)

Haves

• Expiry of the data/keys by TTL
• Transactions
• Pub/Sub
• Distributed (by replication or/and sharding)
• (Distibuted locking)

redis-2-getstarted.md

Get started

• Server (local) C:\Program Files\Redis

• Server AWS Elasticache https://ap-southeast-2.console.aws.amazon.com/elasticache/home?region=ap-southeast-2#redis:

• Client (cmd) C:\Program Files\Redis\redis-cli.exe

SELECT 1
KEYS * 
SET mykey "Hello"
KEYS * 

• Client (desktop)

• C# Client : StackExchange (~= redis-cli)

// your connection manager to redis. Heavy to instanciate (hint: use SingleInstance)
ConnectionMultiplexer redis = ConnectionMultiplexer.Connect("localhost"); 

IDatabase db = redis.GetDatabase(1); 

db.StringSet("mykey", "Hello");
string value = db.StringGet("mykey");

Console.WriteLine(value); // writes: "Hello"

redis-3-datastructures.md

Data Structures

• Everything is a key->value tuple.
• The key is always string
• The value can be of any of the data structures

Strings

"key" -> "door"
"int" -> 1
"dec" -> 0.1

• Fast read by key, fast write
• Expiry on a single key
• Use case: retrieval by unique id

Hashes

"companies" -> { "google" -> "blah", "yahoo" -> "bleh", ... }
"fruits" -> { "banana" -> "yellow", "apple" -> "red", "orange" -> "orange", ... }

• Fast read by (hashset's key, hash enty's key)
• Expiry on the full hashset
• Use case: retrieval by unique id, get a finite collection (get all)

Sets

"tempeature" -> { "fahrenheit", "celsius",  "..." }
"companiesbycategory[it]" -> { "google", "yahoo"  }

• Set operations (union, intersects ... )
• Expiry on the set
• Use case: grouping things

Sorted Sets

"runners" -> [ (john:1), (mary:2),  "..." ]

• Retrieve by rank (score) or by value.
• Expiry on the set
• Use case: indexing by score (date time range etc ...), fetch by slicing

Lists

"posts" -> { "today's post", "yesterday post",  "..." }
"fbwall" -> { ... } 

• Retrieval from head or tail = Java Linkedlist
• Expiry on the set
• Use case: threads, facebook posts ...

redis-4-usecase.md

In Practice (Use case : Companies)

We want a cache which can achieve the following operations over a list of companies (Scanning the keys is not an option).

• GetById(id)
• GetAll()
• GetbyAlias(alias)

Solution1 (using strings, sets)

// getbyid(id)
company:id1 -> { payload1 }
company:id2 -> { payload2 }
company:id3 -> { payload3 }

// get all (variant 1 - with payload)
allcompanies -> ( payload1, payload2, payload3 )

// get all (variant 2 - by id)
allcompanies -> ( company:id1, company:id2, company:id3 )



// get by alias (variant 1 - with payload)
company:alias[BHP] ->  payload1 
company:alias[IBM] ->  payload2 
company:alias[IBM] ->  payload3 

// get by alias (variant 2 - by id)
company:alias[BHP] -> company:id1  
company:alias[IBM] -> company:id2  
company:alias[IBM] -> company:id2  

Solution 2 (using hashsets)

// get by id  & get all
company:master -> { (id1 -> payload1),(id2 -> payload2),(id3 -> payload3) }

// get by alias
company:byalias -> { (BHP -> payload1),(IBM -> payload2),(IBM -> payload3) }

redis-5-transactions.md

Transactions

Redis support transactions, as per definition, a transaction guarantee that no other write operation will occurs during while the transaction is pending.

Simplest demonstration using the C# client

    // start the transaction
	var transaction = _database.CreateTransaction();

	// add operations to the transaction context
	
	// Resharper will complain as the operation bundle in the transaction are awaitable not being explicitly awaited 
	// Using #pragma warning disable 4014 will shut it. 
	transaction.StringSetAsync("myKey1", "val1");
	transaction.StringSetAsync("myKey2", "val2"); 
	
	// if I care about the return (anti-CQS)
	var task = transaction.StringSetAsync("myKey2", "val2");  
	
	// transaction execution
	await transaction.ExecuteAsync();
	
	// now 'task' must be 'completed', and its result will be awailable
	var result = task.Result; // pseudo code. 
	
	

Transactions are per Redis instance. For distributed transactions see distibuted locks.

redis-6-clustering.md

Scalability / clustering features

Sharding scales writes (by splitting the keys amongst the server instances), Replication scale reads by replicating the db (replica = bode)

AWS Replication

On AWS, by scaling up the Redis configuration, the number of nodes will increases, each node represent a replicated copy of the Redis Instance.

AWS Sharding

On AWS, Redis configuration needs to be cluster mode enabled. Sharding can be used on top of replication.

redis-7-pubsub.md

Pub/sub

Redis got built-in a pub/sub messaging mechanism.

• Messages are non-durable
• It has nothing to do with Stroing data in Redis (non-db related)
• Use channels (~= Azure or SQS topics) to manage subscriptions
• This mechanism can be used to listen to Redis own events (keyspace events)

Key Expiry Events (with redis-cli)

Enabling KeySpace notifications (expiry only)

CONFIG SET notify-keyspace-events xKE
CONFIG GET notify-keyspace-events 

All events (makes the whole system really chatty, not recommended for production systems)

CONFIG SET notify-keyspace-events AKE 
CONFIG GET notify-keyspace-events 

Test it with redis-cli (open a listener to event on db 0)

PSUBSCRIBE __keyspace@0__:*   
PSUBSCRIBE __keyevents@<database>__:<command pattern> 

With the StackExchange client

create and open a listener to events on db 0

// Using stackExchange.Redis
using (ConnectionMultiplexer connection = ConnectionMultiplexer.Connect("localhost"))
{
    IDatabase db = connection.GetDatabase();
    ISubscriber subscriber = connection.GetSubscriber();

    subscriber.Subscribe("__keyspace@0__:*", (channel, value) =>
        {
            if ((string)channel == "__keyspace@0__:users" && (string)value == "expriy")
            {
                // Do stuff if some item is added to a hypothetical "users" set in Redis
            }
        }
    );
}

http://www.redis.io/topics/notifications

Keys with a time to live associated are expired by Redis in two ways:

When the key is accessed by a command and is found to be expired. Via a background system that looks for expired keys in background, incrementally, in order to be able to also collect keys that are never accessed.

The expired events are generated when a key is accessed and is found to be expired by one of the above systems, as a result there are no guarantees that the Redis server will be able to generate the expired event at the time the key time to live reaches the value of zero. If no command targets the key constantly, and there are many keys with a TTL associated, there can be a significant delay between the time the key time to live drops to zero, and the time the expired event is generated.

Enabling KeySpace notifications (expiry only)

CONFIG SET notify-keyspace-events xKE
CONFIG GET notify-keyspace-events 

All events (makes the whole system really chatty, not recommended for production systems)

CONFIG SET notify-keyspace-events AKE 
CONFIG GET notify-keyspace-events 

Test it with redis-cli (open a listener to event on db 0)

PSUBSCRIBE __keyspace@0__:*   
PSUBSCRIBE __keyevents@<database>__:<command pattern> 

With the StackExchange client (create and open a listener to events on db 0)

// Using stackExchange.Redis
using (ConnectionMultiplexer connection = ConnectionMultiplexer.Connect("localhost"))
{
    IDatabase db = connection.GetDatabase();
    ISubscriber subscriber = connection.GetSubscriber();

    subscriber.Subscribe("__keyspace@0__:*", (channel, value) =>
        {
            if ((string)channel == "__keyspace@0__:users" && (string)value == "expriy")
            {
                // Do stuff if some item is added to a hypothetical "users" set in Redis
            }
        }
    );
}