Skip to content

Instantly share code, notes, and snippets.

View linuxkidd's full-sized avatar

Michael J. Kidd linuxkidd

35 followers · 10 following
View GitHub Profile
@linuxkidd
linuxkidd / ceph_osd_df.py
Created November 4, 2014 18:05
Get full DF output for all OSDs in a Ceph Cluster from Ceph itself.
#!/usr/bin/env python
import json,sys
from subprocess import Popen, PIPE
from StringIO import StringIO
r = Popen(['ceph','report'],stdout=PIPE,stderr=PIPE)
cephreport, _ = r.communicate()
obj=json.load(StringIO(cephreport))
@linuxkidd
linuxkidd / pgs_per_osd.py
Last active August 29, 2015 14:08
Output OSD size statistics, total PGs per OSD and per-pool PGs per OSD (primary PG count and secondary PG count)
#!/usr/bin/env python
import sys
import simplejson as json
obj=json.load(sys.stdin)
# osd_data_object osd.id = [ 'osdSize': [size,used,avail,used%], 'TotalPGs': [primary, secondary], 'PoolName1': [primary, secondary] , 'PoolName2': [primary, secondary], ... ];
osd_data={}
@linuxkidd
linuxkidd / ceph_osd_dump.py
Last active August 29, 2015 14:08
Duplicate osd dump from ceph report (piped in)
#!/usr/bin/env python
import sys
import simplejson as json
obj=json.load(sys.stdin)
for field in [ 'epoch', 'fsid', 'created','modified']:
print field+' '+str(obj['osdmap'][field]);
@linuxkidd
linuxkidd / getcrushmap.py
Last active August 29, 2015 14:08
Parse 'ceph report' and generate a text version of crushmap
#!/usr/bin/env python
import sys
import simplejson as json
obj=json.load(sys.stdin)
hashes_data={0:'rjenkins1', 'rjenkins1': 0}
ruletypes_data={1:'replicated',2:'raid4',3:'erasure'}
itemById={}
@linuxkidd
linuxkidd / average_multiple_fio.py
Last active August 29, 2015 14:16
#!/usr/bin/python
import sys
import csv
debug=False
stats={}
files=sys.argv
for csvFile in files:
@linuxkidd
linuxkidd / parse_controller.sh
Created March 4, 2015 22:57
#!/bin/bash
echo osd,device,pci id,pci path,description
for i in $(mount | awk '/ceph\/osd/ {printf("%s,%s\n", $1, $3)}'); do
osdid=$(echo $i | awk -F, '{print $2}' | awk -F\/ '{sub (/[^0-9]+/, "", $NF); print "osd."$NF}')
basedev=$(basename $(echo $i | awk -F, '{print $1}' | sed -e 's/[0-9]//g'))
syspath=$(find /sys/devices/ -name $basedev)
hostpath=$(basename $(echo $syspath | awk -F\/host '{print $1}') | sed -e 's/^[0-9]*://')
if [ $(echo $hostpath | grep ata -c) -gt 0 ]; then
hostpath=$(basename $(dirname $(echo $syspath | awk -F\/host '{print $1}')) | sed -e 's/^[0-9]*://')
fi
@linuxkidd
linuxkidd / remove_osd.sh
Created March 6, 2015 15:44
#!/bin/bash
echo -n "Finding host.. "
remosd=$1
remhost=$(ceph osd find $remosd 2>/dev/null | grep host | awk '{print $4}' | sed -e s/[\",]//g)
if [ "$remhost"x == ""x ]; then
echo Failed: osd.$remosd does not exist.
else
echo $remhost
ssh $remhost -- "echo -n 'Stopping osd.$remosd: '; stop ceph-osd id=$remosd; echo -n 'Removing upstart flag: '; rm -f /var/lib/ceph/osd/ceph-$remosd/upstart; echo done; remdrive=\$(mount | grep \"ceph-$remosd \" | awk '{print \$1}'); echo -n \"Umounting $remdrive: \"; umount \$remdrive; echo done; echo -n \"Removing mount point\"; rmdir /var/lib/ceph/osd/ceph-$remosd; echo done"
@linuxkidd
linuxkidd / RV-C__CAN_Bus_Monitor.ino
Created November 23, 2015 19:45
/*
* CAN_Bus_Monitor
* by: Michael J. Kidd <[email protected]>
* Rev: 1.0
* Date: 2015-11-23
*
*/
#include <SPI.h>
#include "mcp_can.h"
@linuxkidd
linuxkidd / led_color.py
Created January 29, 2016 02:03
Python code to change the color on WiFi RGB LED controller
#!/usr/bin/python
import sys
import socket
def Main(host,color):
port = 5577
s = socket.socket()
s.connect((host, port))
@linuxkidd
linuxkidd / map_slow_requests_to_buckets.sh
Last active August 12, 2016 16:19
Provided a ceph.log and ceph osd tree output, this script generates a CSV mapping any 'slow request' entries to their appropriate buckets for identifying common hardware to the slow requests.
#!/bin/bash
if [ -z $2 ]; then
echo
echo "Usage: $(basename $0) <ceph.log> <osd-tree>"
echo
exit 1
fi
if [ ! -e $1 ]; then