Skip to content

Instantly share code, notes, and snippets.

View Kami's full-sized avatar
:octocat:
🐕‍🦺 🦮 🇸🇮 🇺🇦 🇹🇼

Tomaz Muraus Kami

:octocat:
🐕‍🦺 🦮 🇸🇮 🇺🇦 🇹🇼
View GitHub Profile
import os.path
from os.path import join as pjoin
from gevent import monkey
from gevent.pool import Pool
monkey.patch_all()
from libcloud.storage.providers import get_driver
from libcloud.storage.types import Provider
@Kami
Kami / gist:2763687
Created May 21, 2012 18:12
cassandra restore
1. List all the buckets.
2. Find latest manifest in each bucket (sort files by name DESC, get the first one)
3. Open manifest, read the names of the files to download
4. Download the files in like /tmp/<node-name>/<keyspace name>
5. Start restoring sstables for one node at a time sstableloader --debug /tmp/<node-name>/<keyspace name> // todo - sstable loader reads config in ../conf/cassandra.yaml - this has been changed in 1.1 and you can directly pass in a list of initial nodes
@Kami
Kami / gist:2773722
Created May 23, 2012 07:34
cassandra log on sstableloader
TRACE 22:35:55,795 /127.0.0.1local generation 1337553325, remote generation 1337553325
TRACE 22:35:55,795 Updating heartbeat state version to 31 from 29 for /127.0.0.1 ...
DEBUG 22:35:55,795 Schema on /127.0.0.1 is old. Sending updates since 00000000-0000-1000-0000-000000000000
DEBUG 22:35:55,795 collectAllData
DEBUG 22:35:55,798 collecting 0 of 100: ed95e4f0-a2cb-11e1-0000-7fd66bb03adb:false:6861@1337553254704
DEBUG 22:35:55,799 collecting 1 of 100: edb1aa50-a2cb-11e1-0000-7fd66bb03adb:false:7501@1337553254774
DEBUG 22:35:55,802 collecting 2 of 100: edb92460-a2cb-11e1-0000-7fd66bb03adb:false:7882@1337553254824
DEBUG 22:35:55,802 collecting 3 of 100: edc13ab0-a2cb-11e1-0000-7fd66bb03adb:false:8232@1337553254876
DEBUG 22:35:55,802 collecting 4 of 100: edc97810-a2cb-11e1-0000-7fd66bb03adb:false:8581@1337553254932
DEBUG 22:35:55,802 collecting 5 of 100: edd007c0-a2cb-11e1-0000-7fd66bb03adb:false:8939@1337553254974
@Kami
Kami / gist:2773733
Created May 23, 2012 07:38
sstableloader log
kami@lucid:~$ /opt/cassandra/bin/sstableloader KEYSPACE
log4j:WARN No appenders could be found for logger (org.apache.cassandra.config.DatabaseDescriptor).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Starting client (and waiting 30 seconds for gossip) ...
Streaming revelant part of <keyspace>/cf1-hc-248-Data.db <keyspace>/cf2-253-Data.db <keyspace>/cf3-hc-29-Data.db to [/127.0.0.1]
progress: [/127.0.0.1 0/30 (0)] [total: 0 - 0MB/s (avg: 0MB/s)]Exception in thread "Streaming:1" java.lang.RuntimeException: java.net.SocketException: Connection reset
at org.apache.cassandra.utils.FBUtilities.unchecked(FBUtilities.java:689)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
@Kami
Kami / gist:2802590
Created May 27, 2012 07:23
Swiz - serialization example
var swiz = require('swiz');
var O = swiz.struct.Obj;
var F = swiz.struct.Field;
var defs = [
O('Server',
{
'fields': [
F('id', {'src': 'hash_id', 'desc': 'hash ID for the server', 'attribute': true}),
F('is_active', {'src': 'active', 'desc': 'is the server active?', 'coerceTo': 'boolean'}),
@Kami
Kami / gist:2802627
Created May 27, 2012 07:33
Swiz - de-serialization example
var swiz = require('swiz');
var O = swiz.struct.Obj;
var F = swiz.struct.Field;
var defs = [
O('Server',
{
'fields': [
F('id', {'src': 'hash_id', 'desc': 'hash ID for the server', 'attribute': true}),
F('is_active', {'src': 'active', 'desc': 'is the server active?', 'coerceTo': 'boolean'}),
@Kami
Kami / gist:2802631
Created May 27, 2012 07:34
Swiz - de-serialization example (output)
{ id: 15245,
name: 'server #1',
is_active: true,
agent_name: 'some agent',
public_ips: [ '123.45.55.44', '122.123.32.2' ],
state: 'active',
opts:
{ option1: 'defaultval',
option2: 'defaultval',
option3: 'something' },
@Kami
Kami / gist:2802638
Created May 27, 2012 07:39
Swiz - serialization example (output - JSON)
JSON:
{
"id": "15245",
"is_active": true,
"name": "server #1",
"agent_name": "some agent",
"public_ips": [
"123.45.55.44",
"122.123.32.2"
],
@Kami
Kami / gist:2802640
Created May 27, 2012 07:41
Swiz - serialization example (output - XML)
<?xml version="1.0" encoding="utf-8"?>
<server id="15245" name="server #1">
<is_active>true</is_active>
<agent_name>some agent</agent_name>
<public_ips>
<ip>123.45.55.44</ip>
<ip>122.123.32.2</ip>
</public_ips>
<state>active</state>
<opts>
@Kami
Kami / gist:2802676
Created May 27, 2012 07:49
Swiz - validation example
var swiz = require('swiz');
var Valve = swiz.Valve;
var Chain = swiz.Chain;
var O = swiz.struct.Obj;
var F = swiz.struct.Field;
var defs = [
O('Server',
{
'fields': [