Skip to content

Instantly share code, notes, and snippets.

View kf0jvt's full-sized avatar

Kevin Thompson kf0jvt

View GitHub Profile
@kf0jvt
kf0jvt / gist:6470693
Last active December 22, 2015 12:09
Top Ten largest data breaches in the VCDB by record count. #vcdb #javascript
foo = db.vcdb.aggregate([
{ $unwind : "$victim" },
{ $project : { _id : 0,
data_total : "$attribute.confidentiality.data_total",
incident : "$timeline.incident",
victim_id : "$victim.victim_id" } },
{ $sort : { "data_total" : -1 } },
{ $limit : 10 }
]);
<!DOCTYPE html>
<script src="http://mbostock.github.com/d3/d3.v2.js?2.8.1"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.7.2/jquery.min.js"></script>
<style>
body {
font: 10px sans-serif;
}
rect {
@kf0jvt
kf0jvt / date_convert.js
Created September 18, 2013 12:47
After importing json into mongo some date fields may be interpreted as text. This script will go through and convert those fields to dates in mongo.
/usr/bin/mongo yourdbname --eval "db.yourcollectionname.find().forEach(function(doc){doc.yourdatefield = new ISODate(doc.yourdatefield);db.yourcollectionname.save(doc)});"
@kf0jvt
kf0jvt / prc_fix.py
Created September 24, 2013 04:11
Script I wrote to fix a problem that we found in VCDB where a few of the incidents did not have a valid confidentiality attribute. We created new json files, but we couldn't overwrite the old ones because other changes would have been lost. This script maps the reference field to the filename so we can replace just the confidentiality section of…
import os
import json
vcdb_path = 'Documents/development/python/VCDB/incidents'
prc_fix_path = 'Downloads/prc-fix'
mapping = {}
for filename in os.listdir(vcdb_path):
if filename.endswith('.json'):
i = json.loads(open(os.path.join(vcdb_path,filename)).read())
@kf0jvt
kf0jvt / travel-plans.r
Created September 26, 2013 20:47
simple script I use to estimate travel expense for a trip. Modify as needed.
library(mc2d)
flight_min = 323
flight_max = 660
flight_ml - 477
admission = 103.49
hotel_min = 177
hotel_max = 310
@kf0jvt
kf0jvt / vcdb.py
Last active December 24, 2015 22:39
Common stuff that I use in VCDB maintenance
import json
import os
from datetime import datetime
import uuid
# i = getIncident('blahblahblah.json')
def getIncident(inString):
return json.loads(open(inString).read())
# updateIncident(i, 'blahblahblah.json', True)
@kf0jvt
kf0jvt / internal-misuse.py
Created October 8, 2013 20:19
Takes a CSV file and creates json output
import json
import uuid
import copy
import csv
from datetime import datetime
# putIncident(i, 'blackfist')
def putIncident(i, analyst="blackfist"):
i['plus']['analyst'] = analyst
i['plus']['created'] = datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%SZ')
@kf0jvt
kf0jvt / keyczar.py
Created October 14, 2013 02:01
Keyczar example
from keyczar import keyczar
s = 'secret string'
location = '/tmp/kz'
crypter = keyczar.Crypter.Read(location)
s_encrypted = crypter.Encrypt(s)
s_decrypted = crypter.Decrypt(s_encrypted)
print s
print s_encrypted
print s_decrypted
@kf0jvt
kf0jvt / gist:7200902
Created October 28, 2013 17:22
How to use csv.DictReader even when you're dealing with shitty data and header row names are duplicated. First you have to make an array of unique header names and tell the DictReader to use that instead of the first row of the file.
anus = reader(open('bullshit_file.csv','rU'))
fieldnames = []
# Read the first row of the csv file and put all the values into a list. Duplicates will not be overwritten.
for headername in anus.next():
if headername not in fieldnames:
fieldnames.append(headername)
else:
fieldnames.append(headername + " Other")
@kf0jvt
kf0jvt / gist:7219548
Created October 29, 2013 17:57
Using Alanis Morissette to Improve the Effectiveness of Wife Trolling

At any given time I have several research projects going on, but none of them give me greater pleasure than the cutting-edge research I've done in the field of Wife Trolling, the subtle art of irritating your significant other as way to express your love. Here at the Southern Minnesota Research Center I spend countless hours experimenting with new ways to troll my research subject (A.K.A my wife). Recently I've made a breakthrough discovery that I need to share in blog format since Wife Trolling doesn't have a journal (yet).

The Lethal Combination

This new effective technique is based on several assumptions which may not always hold true for any given wife, but are generalizable across the population of North American wives.

  • Women love Alanis Morissette
  • Alanis Morissette has an irritating voice
  • Wives like to remind their husbands of various things they need to do or have forgotten to do. The street term for this is 'nagging' although I prefer the technical term 'feedback.'

The technique

This mor