hivefans

MessagePack update proposal v5

# MessagePack specification

MessagePack is an object serialization specification like JSON.

MessagePack has two concepts: type system and formats. Serialization is conversion from application types into MessagePack formats via MessagePack types. Deserialization is conversion from MessagePack formats into application types via MessagePack types.

Serlialization:

hivefans

import tornado
import tornado.ioloop
import tornado.web
import os, uuid

__UPLOADS__ = "uploads/"

class Userform(tornado.web.RequestHandler):
    def get(self):
        self.render("fileuploadform.html")

hivefans

import tornado.ioloop
import tornado.web
from datetime import datetime
import urlparse
from bson.json_util import dumps

import pymongo
from pymongo import Connection

class Home(tornado.web.RequestHandler):

hivefans

#!/usr/bin/env python
# -*- coding: utf-8 -*-

"""
slideshare-dl.py
~~~~~~~~~~~~~~~~

slideshare-dl is a small command-line program 
for downloading slides from SlideShare.net

hivefans

Introduction to Kafka

Kafka acts as a kind of write-ahead log (WAL) that records messages to a persistent store (disk) and allows subscribers to read and apply these changes to their own stores in a system appropriate time-frame.

Terminology:

• Producers send messages to brokers
• Consumers read messages from brokers
• Messages are sent to a topic

hivefans

require 'zookeeper'

class Lock  
  def initialize(host, root="/my-app")    
    @zk = Zookeeper.new(host)
    @root = root
  end
  
  def with_lock(app, timeout, timeout_callback, &block)
    new_lock_res = @zk.create(:path => "#{@root}/#{app}-", :sequence => true, :ephemeral => true)

hivefans

#!/bin/bash

# Install parallel on CentOS 5.
# Assumes you are root. Prefix w/ sudo if not.

cd /etc/yum.repos.d/
wget http://download.opensuse.org/repositories/home:tange/CentOS_CentOS-5/home:tange.repo
yum install parallel

hivefans

As compiled by Kevin Wright a.k.a @thecoda

(executive producer of the movie, and I didn't even know it... clever huh?)

please, please, please - If you know of any slides/code/whatever not on here, then ping me on twitter or comment this Gist!

This gist will be updated as and when I find new information. So it's probably best not to fork it, or you'll miss the updates!

Monday June 16th

hivefans

####
#  Description:a ruby script to sync consumers offsets with brokers offsets.
#  Requirements: zookeeper
#                sudo gem install zookeeper
#
#####
require 'rubygems'
require 'zookeeper'
require 'socket'

hivefans

#
# Fluent
#
# Copyright (C) 2011 FURUHASHI Sadayuki
#
#    Licensed under the Apache License, Version 2.0 (the "License");
#    you may not use this file except in compliance with the License.
#    You may obtain a copy of the License at
#
#        http://www.apache.org/licenses/LICENSE-2.0