daniel-j-h

Forcing Ceph into mapping all objects to a single PG

Motivation

Recently I was wondering how Ceph (i.e. Rados in particular) maps object to Placement Groups (PGs).

It basically works like the following:

pg = hash(object)

osd = crush(pg)

daniel-j-h

#!/usr/bin/env python2
# -*- coding: utf-8 -*-

# I used this in order to insert the preimages into a local ceph cluster,
# started in my ceph dev directory via src/vstart.sh. Adapt paths if needed.

import sys, os

srcdir = '/home/daniel/cn-work/ceph-dev/src'
pool = 'collisions'

daniel-j-h

#!/usr/bin/env python2
# -*- coding: utf-8 -*-

import sys, json
from collections import defaultdict
from scipy.stats import chisquare

# usage: ceph pg dump_json | ./check_distribution.py

if __name__ == '__main__':

daniel-j-h

#include <thread>
#include <utility>
#include <stdexcept>


// takes ownership of the thread
class scoped_thread {
  public:
    explicit scoped_thread(std::thread t_) : t(std::move(t_)) {
      if(!t.joinable()) throw std::logic_error{"no thread"};

daniel-j-h

#include <atomic>

class spinlock_mutex {
  public:
    spinlock_mutex() : flag(ATOMIC_FLAG_INIT) {}

    void lock() { while(flag.test_and_set(std::memory_order_acquire)); }
    void unlock() { flag.clear(std::memory_order_release); }

  private:

daniel-j-h

// Suppose we have a templated framework and we want to adapt its interface -> use the adapter pattern.


// Our templated framework with awkward get_t and set_t member functions:

template <typename T>
class Framework {
public:
  Framework(T t = {}) : t_{std::move(t)}  { }
  T get_t() const { return t_; }

daniel-j-h

#!/usr/bin/env hy

(import [operator [eq]]
        [itertools [chain count]]
        [functools [partial reduce]])


(defn map-indexed [f coll]
  (map f (count) coll))

daniel-j-h

Introduction To PostGIS

• Blog https://daniel-j-h.github.io/post/gentle-introduction-postgis/
• PostGIS reference: http://postgis.net/docs/reference.html
• Postgres, esp. II.7 Queries http://www.postgresql.org/docs/9.4/interactive/index.html
• geojson.io: http://geojson.io/
• Mapbox: https://www.mapbox.com/
• CartoDB: http://cartodb.com/

daniel-j-h

// See: EMC++, Item 4
template<typename>
class CompileTimeType;

int main() {
  int x;


  CompileTimeType<decltype(x)> x_Type;

daniel-j-h

#include <iostream>
#include <string>
#include <tuple>


// See: EMC++, Item 10
namespace {
  using Person = std::tuple<std::string, std::string, int>;
  enum {                    FirstName,   LastName,    Age };
}