This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import urllib | |
| import re | |
| def ungoogle(url, type="pdf"): | |
| try: | |
| return urllib.unquote(re.search("(?<=url=).+"+type, url).group()) | |
| except: | |
| return urllib.parse.unquote(re.search("(?<=url=).+"type, url).group()) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /* hack to clear screen in stata */ | |
| program define cls | |
| di _newline(`=c(pagesize)') | |
| end |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import numpy as np | |
| import matplotlib.pyplot as plt | |
| #As a path collection | |
| from matplotlib.collections import LineCollection | |
| fig, ax = plt.subplots() | |
| r = np.arange(0, .075, 0.00001) | |
| nverts = len(r) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "metadata": { | |
| "name": "wiki_scrape" | |
| }, | |
| "nbformat": 3, | |
| "nbformat_minor": 0, | |
| "worksheets": [ | |
| { | |
| "cells": [ | |
| { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| """ | |
| Creates a task interface. | |
| Change to | |
| view = rc[:] | |
| for a direct interface. | |
| """ | |
| from IPython.parallel import Client |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import statsmodels.api as sm | |
| from statsmodels.formula.api import ols | |
| # load some data | |
| dta = sm.datasets.longley.load_pandas() | |
| # make a standardized RHS formula | |
| stand = "standardize(%s)" | |
| std_rhs = ' + '.join([stand]*len(dta.exog_name)) % tuple(dta.exog_name) | |
| print std_rhs |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # this is a translation of orthpoly.ado from Stata 11. Their license likely applies. | |
| def orthpoly(X, deg, weights=None): | |
| """ | |
| Christoffel-Darboux recurrence relation for orthogonal polynomials. | |
| """ | |
| nobs = len(X) | |
| orth_poly = np.ones((nobs, deg+1)) | |
| for i in range(1, deg+1): | |
| t = X*orth_poly[:,i-1]**2 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # you can use something like this if read_html fails to find a table | |
| # if you have bs4 >= 4.2.1, you can skip the lxml stuff, the tables | |
| # are scraped automatically. 4.2.0 won't work. | |
| import pandas as pd | |
| from lxml import html | |
| url = "http://www.uesp.net/wiki/Skyrim:No_Stone_Unturned" | |
| xpath = "//*[@id=\"mw-content-text\"]/table[3]" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| function pushdp { | |
| pushd `python -c "import os; import $1; print os.path.dirname($1.__file__)"` | |
| } |