k0emt’s gists

k0emt / Enron_to_Mongo.py

Last active April 26, 2018 04:05

Convert the Enron email dataset files to a MongoDB. Blog post here: http://soloso.blogspot.com/2011/07/getting-enron-mail-database-into.html Brendan McAdams @rit created a version of the code which utilizes the Python email library to produce a database with more metadata. You can see the results of his work here: http://mongodb-enron-email.s3-w…

	import os
	import datetime
	from pymongo import MongoClient

	__author__ = 'k0emt'
	MAIL_DIR_PATH = '/Users/k0emt/Projects/enron/enron_mail_20110402/maildir'
	PREFIX_TRIM_AMOUNT = len(MAIL_DIR_PATH) + 1
	MAX_USER_RUN_LIMIT = 50
	MAX_USER_EMAILS_PER_FOLDER_FILE_LIMIT = 2
	counter = 1

k0emt / Experiment.py

Last active June 11, 2023 15:41

Basic Hello world in Python with corresponding unittest

	__author__ = 'k0emt'

	class Greeter:
	def __init__(self):
	self.message = 'Hello world'
	# print self.message

k0emt / Arguments.py

Created September 2, 2011 20:15

Overloading constructor and variable arguments example

	__author__ = 'k0emt'

	class Greeter:
	def __init__(self):
	self.message = 'Hello world!'
	# print self.message

	class GreeterOverloaded:
	def __init__(self, greeting=None):
	if greeting is None:

k0emt / DarkMatterLogger.py

Created September 15, 2011 04:00

Demo code for RabbitMQ Publish/Subscribe (fanout) exchange with Python

	import sys
	import pika

	# prerequisites are that you have RabbitMQ installed
	# create a "darkmatter" named VirtualHost (VHOST)
	# rabbitmqctl.bat add_vhost darkmatter
	# create a user APP_USER with associated APP_PASS word
	# rabbitmqctl add_user darkmatteradmin <password>
	# give the APP_USER the necessary permissions
	# rabbitmqctl set_permissions -p darkmatter darkmatteradmin "." "." ".*"

k0emt / pymongo_work_with_document.py

Last active December 12, 2015 01:18

MongoDB, PyMongo example working with returned documents.

	from pymongo import MongoClient
	import pymongo
	import sys

	# establish a connection to the database
	connection = MongoClient("mongodb://localhost", safe=True)

	# get a handle to the school database
	db = connection.school
	scores = db.scores

k0emt / updating_docs.py

Last active August 14, 2019 20:51

This gist shows how to loop throw and alter/update an individual uniquely identified document.

	from pymongo import MongoClient
	import sys

	# code example to show updating individual records in a loop

	# initialize the database with:
	# mongoimport -d school -c scores --type json grades.js

	# verify no records with "added"
	# db.scores.find({"added":{$exists:true}}) // returns nothing

k0emt / cities.xml

Created February 15, 2013 07:44

Example of XSLT Transformation with for-each-grouping. command to create: saxon -xsl:for-each-group.xsl -s:cities.xml > grouped_cities.xml

	<cities>
	<state>
	<name>Alabama</name>
	<city>Abbeville</city>
	<number>1</number>
	</state>
	<state>
	<name>Alabama</name>
	<city>Adamsville</city>
	<number>1</number>

k0emt / mongoGetKeys.py

Last active December 13, 2015 22:18

Utility code to get the keys from a document in a specified MongoDB database and collection.

	#!/usr/local/bin/python

	import sys
	from pymongo import MongoClient

	if len(sys.argv) < 3:
	print 'usage is: ' + sys.argv[0] + ' databaseName collectionName'
	sys.exit()

	dbName = sys.argv[1]

k0emt / hello_world_bottle.py

Created April 6, 2013 04:50

Example of hello world in bottle with python

	import bottle

	@bottle.route('/')
	def home_page():
	return "Hello World\n"

	@bottle.route('/testpage')
	def test_page():
	return "this is a test page"

k0emt / create_replica_set.bat

Last active November 20, 2016 18:16

Helper batch file for starting up a replica set named m101. Drop the --logpath option if you want to enjoy the log text flying by in the console. The --oplogSize option is the magic sauce to avoid creation of numerous 512MB+ files up to 5% of disk space.

	mkdir \data\rs1 \data\rs2 \data\rs3
	start mongod --replSet m101 --logpath "1.log" --dbpath \data\rs1 --port 27017 --smallfiles --oplogSize 64
	start mongod --replSet m101 --logpath "2.log" --dbpath \data\rs2 --port 27018 --smallfiles --oplogSize 64
	start mongod --replSet m101 --logpath "3.log" --dbpath \data\rs3 --port 27019 --smallfiles --oplogSize 64

Bryan Nehl k0emt