Skip to content

Instantly share code, notes, and snippets.

{"executionTime":"2013-07-09 04:59:02 PM","stationBeanList":[{"id":5,"stationName":"State St & Harrison St","availableDocks":14,"totalDocks":19,"latitude":41.8739580629,"longitude":-87.6277394859,"statusValue":"In Service","statusKey":1,"availableBikes":5,"stAddress1":"State St & Harrison St","stAddress2":"","city":"","postalCode":"","location":"620 S. State St.","altitude":"","testStation":false,"lastCommunicationTime":null,"landMark":"30"},{"id":13,"stationName":"Wilton Ave & Diversey Pkwy","availableDocks":13,"totalDocks":19,"latitude":41.93250008,"longitude":-87.65268082,"statusValue":"In Service","statusKey":1,"availableBikes":3,"stAddress1":"Wilton Ave & Diversey Pkwy","stAddress2":"","city":"Chicago","postalCode":"","location":"2790 N.Wilton Ave","altitude":"","testStation":false,"lastCommunicationTime":null,"landMark":"66"},{"id":14,"stationName":"Morgan St &18th St","availableDocks":11,"totalDocks":15,"latitude":41.85807921,"longitude":-87.65140281,"statusValue":"In Service","statusKey":1,"availableB
@hunterowens
hunterowens / gist:6174523
Last active December 20, 2015 18:18
DSSG Git Repo Checklist

###Directory Structure

  • Clean, Well named set of directories. Examples include webapp, database, and models.
  • No random files in the root.
  • Explanation of each directory in README.
  • Sub-README's in appropriate folders
  • No directories named after DSSG specific info (ie, person names)
  • Should your team have more than one project, each should have it its own repo.
  • In your data or database folder, provide a way to re-create your database from scratch. .sql files are often appropriate for this.
@hunterowens
hunterowens / gist:6235144
Last active December 21, 2015 02:29
Wiki Guide

REQUIRED WIKI SECTIONS

  • Homepage

    Intro: have a sentence or two about the project: the problems its solving, the partner, and how you're solving it. Also say that "this wiki is the central place to learn about the social problem we worked on, the data we used, the methods we used to solve it, and our findings" so people know what they're looking at. List of pages in the wiki

  • Problem

    An in-depth description of the problem your organization, the problem you're trying to solve, and any relevant domain knowledge. Feel free to copy from blog posts and posters, if relevant.

  • Data

# -----------------------------
# PostgreSQL configuration file
# -----------------------------
#
# This file consists of lines of the form:
#
# name = value
#
# (The "=" is optional.) Whitespace may be used. Comments are introduced with
# "#" anywhere on a line. The complete list of parameter names and allowed
# PostgreSQL Client Authentication Configuration File
# ===================================================
#
# Refer to the "Client Authentication" section in the PostgreSQL
# documentation for a complete description of this file. A short
# synopsis follows.
#
# This file controls: which hosts are allowed to connect, how clients
# are authenticated, which PostgreSQL user names they can use, which
# databases they can access. Records take one of these forms:
@hunterowens
hunterowens / speedtest.py
Last active August 29, 2015 13:55
speedtest.py
import timeit
import pandas as pd
import requests
S3_URL = "https://s3-us-west-2.amazonaws.com/dssgtest01/crimedata.csv"
S3_PATH = "s3://dssgtest01/crimedata.csv"
LOCAL_EBS_PATH = "/mnt/vol-2a96ab12/crimedata.csv"
NFS_PATH = "/mnt/data1/SpeedTest/crimedata.csv"
#S3 Testing. using HTML
@hunterowens
hunterowens / renmin.csv
Last active August 29, 2015 13:59
Chinese Internet Tests
type_of_connection ping download upload
CMCC Paid China Moible 0 26.35 32.36
Renmin WIFI 16 6.24 1.59
Personal WIFI 2 42.96 49.02
Personal WIFI+cVPN 453 1.01 0.007
Personal WIFI+OpenVPN/Digital Ocean 443 0.05 0.33
Personal WIFI+IP VanishVPN 437 0.59 0.08
@hunterowens
hunterowens / lunch.md
Last active August 29, 2015 14:03
JuliaConLunch
  1. Portillos - Classic Chicago - Italian Beef, Hot Dogs - Farther Walk- 100 W Ontario St
  2. Food Trucks - Right outside. Take a Look.
  3. Food For Thought - Food Court/Salad Bar - 401 North Michigan Ave - walk towards Michigan.
  4. Billy Goat Tavern - 430 N. Michigan Ave - LOWER LEVEL- The Classic Chicago Burger Place. Under Tribune Tower.
  5. Water Tower Place Food Court - It's a Mall with a food court. Walk up Michigan.
  6. Epic Burger - New Burger place - 227 E Ontario St
  7. Sayat Nova - Sit down Armenian food. 157 E. Ohio St.
  8. Chipotle - 291 E Ontario St
  9. Potbelly - Local sandwich chain- 277 E Ontario St
  10. Naf Naf Grill - Middle Eastern Chipotle - 326 N Michigan Ave
@hunterowens
hunterowens / connect_and_load.py
Last active December 9, 2023 16:27
Pandas and MSSQL
import pymssql
import pandas as pd
## instance a python db connection object- same form as psycopg2/python-mysql drivers also
conn = pymssql.connect(server="172.0.0.1", user="howens",password="some_fake_password", port=63642) # You can lookup the port number inside SQL server.
## Hey Look, college data
stmt = "SELECT * FROM AlumniMirror..someTable"
# Excute Query here
df = pd.read_sql(stmt,conn)
@hunterowens
hunterowens / xml2csv.py
Created November 29, 2014 18:57
XML->CSV
import xmltodict
import json
import pandas as pd
file = open('/path/to/file.xml','r')
str = file.read()
dictionary = xmltodict.parse(str)
df = pd.DataFrame(dictionary)
# Do any parsing of the XML file here, as you'll want to confirm that you are writing the correct info to csv.
df.to_csv('/path/to/output.csv')