Skip to content

Instantly share code, notes, and snippets.

View paul-english's full-sized avatar

Paul English paul-english

View GitHub Profile
@paul-english
paul-english / EditDistance.scala
Created July 27, 2017 18:08
EditDistance.scala, not the best for prod, spark sql has levenshtein built in, but useful as a template for spark ml transformers
import org.apache.spark.ml.Transformer
import org.apache.spark.ml.param.{ ParamMap, Param }
import org.apache.spark.ml.util.{ Identifiable, DefaultParamsWritable, DefaultParamsReadable }
import org.apache.spark.sql.functions.{ udf, col }
import org.apache.spark.sql.types.{ StructType, StructField, DoubleType }
import org.apache.spark.sql.{ Dataset, DataFrame }
object EditDistance extends DefaultParamsReadable[EditDistance] {
override def load(path: String): EditDistance = super.load(path)
#!/usr/local/bin/python
# !/usr/bin/env python
from __future__ import division, print_function
import random
import sys
from datetime import datetime
import numpy as np
import slackweb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@paul-english
paul-english / code_review.md
Last active April 20, 2017 03:49
Code Review Checklists

General

  • Does this cover any code you've personally worked on in the past 6 months? -- If so, you'll want to do a more thorough review to ensure it doesn't cause you any later headaches
  • Will you have to work on this code or area of the codebase in the future? -- Also a good reason to do a more thorough review of the code base.
  • Is the code cleanly formatted? (whitespace, code organization, commenting, etc)
  • Do the tests pass?
  • Is there commented code that can be removed?
  • If function calls are removed, do those functions get called anywhere else? and if not can the function itself be removed?
pub fn int_from_bytes(x: &[u8]) -> i64 {
from_utf8(x).unwrap().parse::<i64>().unwrap()
}
pub fn float_from_bytes(x: &[u8]) -> f64 {
from_utf8(x).unwrap().parse::<f64>().unwrap()
}
named!(pub number<&[u8], i64>,
do_parse!(opt!(multispace) >>
@paul-english
paul-english / asana.py
Created January 4, 2017 16:26
Asana Python Social Auth
import json
import urllib.error
import urllib.parse
import urllib.request
from social.backends.oauth import BaseOAuth2
class AsanaOAuth2(BaseOAuth2):
"""Asana OAuth authentication backend"""
@paul-english
paul-english / fake_people.py
Created December 21, 2016 18:30
fake_people.py
#!/usr/bin/env python
import random
import string
import numpy as np
import pandas as pd
from faker import Faker
from tqdm import tqdm
#!python
#cython: boundscheck=False
#cython: wraparound=False
#cython: cdivision=True
cimport cython
import sys
from collections import namedtuple
import numpy as np
def offensive_score(player):
score = 0
score += player['passing_yds']*0.04
score += player['passing_tds']*4
score += player['passing_int']*-1
score += player['rushing_yds']*0.1
score += player['rushing_tds']*6
score += player['receiving_rec']*0.5
score += player['receiving_yds']*0.1
score += player['receiving_tds']*6
macro_rules! caseless_tag (
($i:expr, $inp: expr) => (
{
#[inline(always)]
fn as_lower(b: &str) -> String {
let s = b.to_string();
s.to_lowercase()
}
let expected = $inp;