Skip to content

Instantly share code, notes, and snippets.

@edenhill
edenhill / librdkafkastats.json
Created October 16, 2013 22:03
librdkafka statistics output
{
"ts": 268229898540,
"time": 1381960888,
"replyq": 0,
"brokers": [
{
"name": "localhost:9092/bootstrap",
"nodeid": -1,
"state": "UP",
"outbuf_cnt": 0,
@edenhill
edenhill / gist:7597127
Created November 22, 2013 09:17
librdkafkastats.json without arrays. Also includes rtt
{
"ts":22814656926,
"time":1385111797,
"replyq":0,
"brokers":{
"eden.local:9092/bootstrap":{
"name":"eden.local:9092/bootstrap",
"nodeid":-1,
"state":"UP",
"outbuf_cnt":0,
@edenhill
edenhill / gist:8470811
Created January 17, 2014 09:51
WIFEXIT
/**
* Returns a human readable process exit reason based on the exit code.
*/
const char *exec_exitstatus (int status) {
static __thread char ret[128];
if (WIFEXITED(status)) {
if (WEXITSTATUS(status) == 127)
snprintf(ret, sizeof(ret),
"could not execute command");
@edenhill
edenhill / clang -Os output
Created February 11, 2014 02:34
gcc vs clang constant math compilation
clang 3.4-1
0000000000000000 <hashit>:
0: b8 7e f5 35 6d mov $0x6d35f57e,%eax
5: c3 retq
@edenhill
edenhill / gist:30885d7e6781d6a43f2e
Last active August 29, 2015 14:02
osx 10.9 static linking problems with -g
Creating shared library librdkafka.so.1
gcc -g -shared -dynamiclib -Wl,-install_name,/usr/local/lib/librdkafka.so.1 rdkafka.o rdkafka_broker.o rdkafka_msg.o rdkafka_topic.o rdkafka_defaultconf.o rdkafka_timer.o rdkafka_offset.o rdcrc32.o rdgz.o rdaddr.o rdrand.o rdthread.o rdqueue.o rdlog.o snappy.o -o librdkafka.so.1 -lpthread -lz
Creating static library librdkafka.a
ar rcs librdkafka.a rdkafka.o rdkafka_broker.o rdkafka_msg.o rdkafka_topic.o rdkafka_defaultconf.o rdkafka_timer.o rdkafka_offset.o rdcrc32.o rdgz.o rdaddr.o rdrand.o rdthread.o rdqueue.o rdlog.o snappy.o
$ ranlib librdkafka.a
$
..
@edenhill
edenhill / phpkafka.php
Created June 12, 2014 21:28
PHP Kafka new API
# Kafka configuration (https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md)
$config = { "metadata.broker.list" => "broker1.example.com", "socket.timeout.ms" => 30000, ... };
# Create Kafka Producer object.
# Brokers are specified through $config, but we could make it easier for people by having the first
# argument be brokers as well, and rdkafka will use all brokers specified?
$producer = new Kafka::Producer([$brokers,]? $config);
@edenhill
edenhill / gist:de786a98ec6e120020a4
Last active August 29, 2015 14:06
mklove conditional SRCS
include Makefile.config
SRCS_$(WITH_MYOPTION) += myoption.c
SRCS = standard.c foo.c \
$(SRCS_y)
...
#!/usr/bin/python
#
class Burton (object):
def __init__ (self):
self.foo = 1
def __enter__ (self):
return self
#if !DEBUG_SHARED_PTR
/**
* The non-debug version of shared_ptr is simply a reference counting interface
* without any additional costs and no dereferencing.
*/
#define rd_shared_ptr_t RD_UNUSED void
#define rd_shared_ptr_get(OBJ,REFCNT) \
(rd_refcnt_add(REFCNT), (rd_shared_ptr_t *)(OBJ))
Data is always possibly outdated in distributed systems (including multi-threaded applications):
* Dont use atomics, they are slow.
* Use locks, they are fast, especially when uncontended.
* Hold locks as shortly as possible: Lock, copy, unlock, process, propagate, progress
* Strive for thread local data (not TLS) which requires no locking, use inter-thread events to propagate information.
* Process events best-effortly
* Dont demand exactness, but require progress. Try to recover.
* Use event version barriers to throw away outdated async data