cdfox’s gists

cdfox / twitter_lsi.py

Created June 17, 2011 05:29 — forked from anonymous/twitter_lsi.py

Applying LSI to Twitter search


	import json
	import urllib
	import urllib2
	from gensim import corpora, models, similarities
	import logging
	import sys
	from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas
	from matplotlib.figure import Figure

cdfox / gist:1031054

Created June 17, 2011 08:07

twitter lsi test with query "haskell"

	Topic 0:

	ICFP Programming Contest 2011 (Lambda: The Gathering) http://bit.ly/mfpZk6 #fp #fsharp #ocaml #haskell #scala #lisp #racket #scheme
	[(0, 0.95871468032041129), (1, 0.079066183724039785)]

	RT @rickasaurus: ICFP Programming Contest 2011 (Lambda: The Gathering) http://bit.ly/mfpZk6 #fp #fsharp #ocaml #haskell #scala #lisp #racket #scheme
	[(0, 0.99229220887802994), (1, 0.062785061633268899)]

	RT @rickasaurus: ICFP Programming Contest 2011 (Lambda: The Gathering) http://bit.ly/mfpZk6 #fp #fsharp #ocaml #haskell #scala #lisp #racket #scheme
	[(0, 0.99229220887802994), (1, 0.062785061633268899)]

cdfox / gist:1031073

Created June 17, 2011 08:31

twitter lsi test with query "haskell", 3 topics

	Topic 0:

	ICFP Programming Contest 2011 (Lambda: The Gathering) http://bit.ly/mfpZk6 #fp #fsharp #ocaml #haskell #scala #lisp #racket #scheme
	[(0, 0.96213088834451832), (1, -0.053502204689306387), (2, -0.07951428719352148)]

	RT @rickasaurus: ICFP Programming Contest 2011 (Lambda: The Gathering) http://bit.ly/mfpZk6 #fp #fsharp #ocaml #haskell #scala #lisp #racket #scheme
	[(0, 0.99540683458237178), (1, -0.034752233550864223), (2, -0.065158846734593753)]

	RT @rickasaurus: ICFP Programming Contest 2011 (Lambda: The Gathering) http://bit.ly/mfpZk6 #fp #fsharp #ocaml #haskell #scala #lisp #racket #scheme
	[(0, 0.99540683458237178), (1, -0.034752233550864223), (2, -0.065158846734593753)]

cdfox / gist:1037261

Created June 21, 2011 04:44

Twitter LSI


	import json
	import urllib
	import urllib2
	from gensim import corpora, models, similarities
	import logging
	import sys

	# First 20 Twitter search results for "python"
	tweets = [u'Long Integer Objects \u2014 Python v2.7.2 documentation http://bit.ly/jEzUVi',

cdfox / gist:2560930

Created April 30, 2012 18:39

Using the Requests library

	import requests
	import codecs

	url = 'http://www.instapaper.com/m'
	params = {'u': 'http://techcrunch.com/2012/04/30/bn-8-k-microsoft-paying-180m-advance-on-nook-for-windows-8-125m-for-content-tech-acquisition/'}
	r = requests.get(url, params=params)

	# r.text holds the response from the server. It appears to be unicode in this case.
	f = codecs.open("foo.html", mode="w", encoding="UTF-8")
	f.write(r.text)

cdfox / gist:2830517

Created May 29, 2012 20:30

hello world

print 'hello world'

cdfox / gist:2910560

Created June 11, 2012 15:09

sentences = [s for rev in reviews for s in rev.split(".")]

cdfox / preprocess.go

Last active December 15, 2015 21:29

Preprocess a set of documents, one on each line of the input file, for LDA inference.

	// For each line of the input file, remove nonalphanumeric characters,
	// lowercase all letters, remove stopwords, and write the result to the output
	// file. I used the answer here as a template for reading/writing files:
	// http://stackoverflow.com/questions/1821811/how-to-read-write-from-to-file/9739903#9739903

	package main

	import (
	"bufio"
	"fmt"

cdfox / gist:6857081

Created October 6, 2013 18:00

Counting the possible orderings of distinct elements [a,b,c,d,e,f] given that a < d, b < e, c < b, c < f, d < f, d < e.

	from itertools import permutations

	perms = permutations(range(6))

	def check(p):
	i1 = p.index(0) < p.index(3)
	i2 = p.index(1) < p.index(4)
	i3 = p.index(2) < p.index(1)
	i4 = p.index(2) < p.index(5)
	i5 = p.index(3) < p.index(4)

cdfox / sqrt_floating_point.py

Created September 19, 2014 15:20

	import math

	def sqrt(x, high, low, e):
	mid = (high + low) / 2.0
	square = mid * mid
	if (square >= x and square - x <= e) or (square < x and x - square <= e):
	return mid
	elif square > x:
	return sqrt(x, high, mid, e)
	else:

Christopher Fox cdfox