Aleck Landgraf alecklandgraf

Python Engineer Take Home Project

Building Energy strives to help businesses and governments understand where energy usage occurs, and how to improve energy efficiency. Let’s look at some real-world data in a similar field; check out World Bank Data on Carbon Emissions. Click "Download Data" and select the format you want to work with, (CSV might be simplest).

Let’s build a simple application to read in this data from the commandline and sort the rows by a function applied across the columns of said rows.

We should be able to answer the following questions:

Sort countries according to the average change in carbon emissions per capita (ascending, descending, etc.).

Elastic{ON} 2015 best practices

Notes by: Aleck Landgraf

config and index settings

#1 take away: use 3 master nodes
- these can be whimpy EC2 instances
- don't issue search queries to them
- don't put data on them, i.e. don't index to them

	<!DOCTYPE html>
	<html>
	<head>
	<title>Chernoff Smileys</title>
	<script type="text/javascript" src="http://mbostock.github.com/d3/d3.js?2.5.0"></script>
	<script type="text/javascript" src="http://mbostock.github.com/d3/d3.layout.js?2.5.0"></script>
	<style type="text/css">

	</style>
	</head>

	#!/usr/bin/env python
	# encoding: utf-8
	"""
	Invalidate Johnny Cache's cache for given models outside "normal Django"
	(e.g. in a management command).
	"""

	from johnny.cache import invalidate
	from johnny.middleware import QueryCacheMiddleware

	import requests


	def do_session_get():
	session = requests.session()
	return session.get('foo')

	#!/bin/sh
	# Just copy and paste the lines below (all at once, it won't work line by line!)
	# MAKE SURE YOU ARE HAPPY WITH WHAT IT DOES FIRST! THERE IS NO WARRANTY!

	function abort {
	echo "$1"
	exit 1
	}

	set -e

	try:
	import xlrd

	def XLSDictReader(f, sheet_index=0):
	data = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)
	book = xlrd.open_workbook(file_contents=data)
	sheet = book.sheet_by_index(sheet_index)

	def item(i, j):
	return (sheet.cell_value(0,j), sheet.cell_value(i,j))

	description "uWSGI server for electris CMS"

	start on runlevel [2345] # start on all runlevels.
	stop on runlevel [!2345] # stop when shutting down.

	respawn # respawn if job crashes or is stopped ungracefully.
	env DEPLOYMENT_TARGET=production # set any environment variables you like here.
	env DJANGO_SETTINGS_FILE=conf/settings.py # more environment variables if you like.
	env PYTHONPATH=/home/ubuntu/apps/my_app:/home/ubuntu/.virtualenv/my_app

	def reindex_orgs(org_ids):
	from seed.services import es_search_query
	from elasticsearch import Elasticsearch
	s = es_search_query()
	s = s.filter('terms', building_snapshot__super_organization=org_ids)
	from seed.services import elasticsearch_client
	es = elasticsearch_client()
	es.delete_by_query(index=settings.ES_INDEX, doc_type='canonical_building', body=s.to_dict())
	from seed.utils.seed_elasticsearch import canon_serializer
	import itertools