Skip to content

Instantly share code, notes, and snippets.

View marians's full-sized avatar

Marian Steinbach marians

145 followers · 12 following
View GitHub Profile
@marians
marians / lang_detection_server.pl
Created March 19, 2013 12:41
This is an HTTP service for natural language guessing of input texts. Run it as "perl lang_detection_server.pl" and open a URL like http://localhost:8080/?text=This+is+just+a+test+string . TextCat source code courtesy of Gertjan van Noord.
# This is a sloppy HTTP server version of TextCat, an n-gram based natural language guesser
# written by Gertjan van Noord in 1997.
# More info: http://odur.let.rug.nl/~vannoord/TextCat/
#
# TextCat was distributed under the GNU Lesser General Public License.
#
# You need the language model files (LM folder from Gertjan's distribution) in a directory.
# Set the variable $opt_d to point to that directory.
@marians
marians / save_tweets.py
Created March 12, 2013 13:20
Save tweets containing certain keywords from the twitter Straming API to MongoDB
import tweetstream
from pymongo import MongoClient
# look for these words:
WORDS = ['word1', 'word2']
TWITTER_USER = ""
TWITTER_PASS = ""
MONGO_DB = 'tweetstream'
@marians
marians / webserver.py
Created October 24, 2012 09:53
A simple development webserver for the console that features throttling (if enabled by -t) and ignores query strings
# encoding: utf-8
"""
A simple development webserver for the console that features
throttling (if enabled by -t) and ignores query strings.
"""
import SimpleHTTPServer
import SocketServer
import os
@marians
marians / Mapping
Created October 17, 2012 10:26
ElasticSearch - documents with multiple geo_point properties
{
spatialtest: {
document: {
properties: {
location: {
type: "geo_point"
},
title: {
type: "string"
}
@marians
marians / gist:2522491
Created April 28, 2012 22:46
Cache-Control: public, max-age=43200
Connection: keep-alive
Content-Length: 81443
Content-Type: text/css; charset=utf-8
Date: Sat, 28 Apr 2012 22:37:56 GMT
ETag: "flask-1335630896.86-81443-1985555092"
Expires: Sun, 29 Apr 2012 10:37:56 GMT
Last-Modified: Sat, 28 Apr 2012 16:34:56 GMT
Server: nginx/0.7.65
Set-Cookie: session="IZK844h3a4CHUU02CIjgyc08RHM=?lang=Vml0CnAxCi4="; Path=/; HttpOnly
@marians
marians / db.sql
Created February 20, 2012 11:11
Collecting tweets mentioning given keywords, storing the result to a MySQL table
CREATE TABLE `tweets` (
`id` varchar(24) NOT NULL DEFAULT '',
`created_at` datetime NOT NULL,
`user_id` bigint(20) unsigned NOT NULL,
`user_name` varchar(128) NOT NULL DEFAULT '',
`user_followers` int(11) unsigned NOT NULL,
`user_friends` int(10) unsigned DEFAULT NULL,
`user_listed` int(10) unsigned DEFAULT NULL,
`user_statuses` int(10) unsigned DEFAULT NULL,
`user_location` varchar(100) DEFAULT NULL,
@marians
marians / 292011.txt
Created February 16, 2012 18:39
OCR-Ergebnisse mit tesseract-ocr (aktuelle SVN-Version vom 16.2.2012)
CDU
CDU-Fraktion in der Bezirksvertretung
des Stadtbezirke Ehrenfeld
Henn Oberbürgennei „ _ ‚ erm Bezirksbürgenneister
Jürgen Roters * " _ „_ . ‚ osef Wrrges
Rathaus Eingaig 28 Feb 2m Bezirksrathaus Ehrenfeld
@marians
marians / kill-old-processes.py
Created January 23, 2012 20:12
A rather simple script to kill stalled processes. Adapt the grep string in the PS_CMD variable to make it match your needs.
# coding: utf-8
"""
Dieses Script beendet bestimmte Scraper-Prozesse, die
schon länger als einen Tag laufen.
"""
PS_CMD = 'ps -eo pid,etime,cmd|grep scrape_updated_dose_values.py'
@marians
marians / getpostcounts.py
Created January 5, 2012 10:26
de. Newsgroups und Statistiken darüber
#!/usr/bin/env python
# encoding: utf-8
"""
This script acquires statistics on usenet groups.
It first reads a list of groups from a file and then
reads the Google Group info page about that group to
gather monthly post counts.
"""
@marians
marians / usenet_group_stats.py
Created January 2, 2012 08:32
#!/usr/bin/env python
# encoding: utf-8
"""
This script acquires statistics on usenet groups.
It first reads a list of groups from one or more usenet servers
and then gets monthly post statistics about these groups from
Google Groups.
"""