- This is the entire internet in thirty minutes!
- First the caveats.
- I am not a network engineer. I write terrible Python and JavaScript for a public radio station. Much of what I say here is apocryphal and possibly entirely wrong. Please do not rely on this information for anything important or you might catch on fire.
- Good, let's get started. What happens when you load an URL? This is is what we're going to do: We're going to trace a request to a Web site as it happens. But first, we need to cover some basics.
- A client is a computer which makes requests.
- A server is a computer which responds to requests.
- The internet is a global network of physically connected servers and clients.
- The World Wide Web is a subset of the internet concerned with hypertext (links) between resources (URLs).
- We got it? Good. Time to really get down to business.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Show hidden characters
{ | |
"auto_complete": false, | |
"close_windows_when_empty": false, | |
"color_scheme": "Packages/Theme - Flatland/Flatland-Monokai.tmTheme", | |
"draw_white_space": "all", | |
"find_selected_text": true, | |
"flatland_square_tabs": true, | |
"fold_buttons": false, | |
"folder_exclude_patterns": | |
[ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
## Generate a full season's worth of batting Marcel projections from past years' stats | |
from createTuple import createTuple ## gist: 778481 | |
from writeMatrixCSV import writeMatrixCSV ## gist: 778484 | |
def makeBatTable(r): | |
for stat in ['AB', 'H', 'D', 'T', 'HR', 'SO', 'BB', 'SF', 'HP', 'CI']: | |
if stat in r: pass | |
else: r[stat] = 0 | |
if r['AB'] == 0: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
## Generate a full season's worth of pitching Marcel projections from past years' stats | |
from createTuple import createTuple ## gist: 778481 | |
from writeMatrixCSV import writeMatrixCSV ## gist: 778484 | |
def makePitTable(r): | |
for stat in ['AB', 'H', 'D', 'T', 'HR', 'SO', 'BB', 'SF', 'HP', 'CI', 'IPouts', 'R']: | |
if stat in r: pass | |
else: r[stat] = 0 | |
ab = 0.9*r['IPouts'] + r['H'] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<!DOCTYPE html> | |
<html lang="en"> | |
<head><meta charset="utf-8"></head> | |
<body> | |
<!-- This is the Underscore.js template for each of our links. --> | |
<!-- I didn't HAVE to make this a template. But I wanted to show --> | |
<!-- how you might do this if you were doing it by our standards. --> | |
<!-- Inside this script tag, you can write HTML with some little things, --> | |
<!-- <%= foo %> means to print the Javascript variable "foo". --> | |
<!-- <% %> means you would like to write some Javascript to be evaluated but not printed. --> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import csv | |
import urllib | |
from operator import itemgetter | |
from collections import defaultdict | |
from os.path import dirname, join | |
# Download CSV of fake Virginia election results to root of project | |
url = "https://docs.google.com/spreadsheet/pub?key=0AhhC0IWaObRqdGFkUW1kUmp2ZlZjUjdTYV9lNFJ5RHc&output=csv" | |
filename = join(dirname(dirname(__file__)), 'fake_va_elec_results.csv') |
I hereby claim:
- I am jeremyjbowers on github.
- I am jeremyjbowers (https://keybase.io/jeremyjbowers) on keybase.
- I have a public key whose fingerprint is C4FC 866A 3227 40F8 DCA4 9432 1900 9AB4 CCBE A520
To claim this, I am signing this object:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<!DOCTYPE html> | |
<html lang="en"> | |
<head> | |
<meta charset="utf-8"> | |
<meta http-equiv="X-UA-Compatible" content="IE=edge"> | |
<meta name="viewport" content="width=device-width, initial-scale=1"> | |
<title>NPR Softball 2014</title> | |
<!-- Bootstrap --> | |
<link rel="stylesheet" href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css"> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env python | |
import json | |
from bs4 import BeautifulSoup | |
import requests | |
BASE_URL = "http://bpldb.bplonline.org/db/formProc/coalmine?firstname=&search_lastname_parameter=starts&lastname=&title=coalmine&mine=&race=&search_firstname_parameter=starts&year=&report=&cause=&occupation=&atnum=%s" | |
def get_count(): |