Skip to content

Instantly share code, notes, and snippets.

View hancush's full-sized avatar
🍓

hannah cushman garland hancush

🍓
View GitHub Profile
  1. Assemble web_info by iterating over table cells parsed from Legistar GUI

  2. Assemble terms by iterating over City Council terms from Legistar API, adding members to the web_info dict as we go

  3. Assemble members by iterating over terms, storing names as keys, and Person objects, loaded with memberships for each City Council term, as values

  4. Continue to assemble members by iterating over body offices for each committee, storing memberships on the appropriate Person objects (or minting

from make_a_formset import formset
# populate your formset
for form in formset:
form_data = form.clean_data
# {'name': 'Hannah', 'favorite_color': 'yellow'}
# do stuff with the data

Where test.txt contains:

a string      with some  extra spaces




line
# header
here's what this is
<<>>=
import pandas as pd
@
<<>>=
data = pd.read_csv('path/to/your/data') # returns a dataframe
@hancush
hancush / django-store-files-s3.md
Last active March 7, 2018 15:53
store django model-related files in s3

Install django-storages:

pip install django-storages

Add to settings:

INSTALLED_APPS = (
TABLES=$(mdb-tables /path/to/YOUR_ACCDB.accdb); \
for TABLE in $TABLES; \
do mdb-export /path/to/YOUR_ACCDB.accdb $TABLE > /path/to/$TABLE.csv; \
done;
from django.db.models import Q
condition = Q()
employers = Employer.objects.all()
if params:
if params.get('name'):
employers = employers.annotate(search_vector=SearchVector('name'))

🤖 Make some data

Your mission

Write a Makefile that gets data from Data.gov and creates a CSV of the 50 most recent datasets uploaded. For each dataset, the output should include:

  • The name of the dataset
  • The date it was uploaded
  • A download link
# usage: python interval.py /path/to/your/file.csv
import csv
import sys
import re
def days_from_interval(interval):
if interval == 'never':
return None
Employer ID
City of Chicago 16
Chicago 16
Metropolitan Pier and Exposition Authority 53
Metropolitan Pier Exposition Authority 53
Third Lake 364
Village Of Third Lake 364
Fox Valley Park District 519
Fox Valley Park District 519
Homewood Flossmoor Park District 541