Skip to content

Instantly share code, notes, and snippets.

View uolter's full-sized avatar

Walter Traspadini uolter

View GitHub Profile
from datetime import datetime
def time_average(time_1, time_2):
"""
return a the average time between time_1 and time_2
as a string in the same format as the input
"""
FMT = '%M:%S:%f'
@uolter
uolter / tf_idf.py
Created December 18, 2013 09:42
Simple term frequency and inverse document implementation
# -*- coding: utf-8 -*-
# <codecell>
# term frequency
from math import log
# XXX: Enter in a query term from the corpus variable
# QUERY_TERMS = ['mr.', 'green']
@uolter
uolter / setup-web2py-ubuntu.sh
Created January 26, 2014 15:00
Set up web2py to run on Ubuntu with apache and mod_wsi
echo "This script will:
1) install all modules need to run web2py on Ubuntu/Debian
2) install web2py in /home/www-data/
3) create a self signed sll certificate
4) setup web2py with mod_wsgi
5) overwrite /etc/apache2/sites-available/default
6) restart apache.
You may want to read this script before running it.
#! /bin/sh
### BEGIN INIT INFO
# Provides: elasticsearch
# Required-Start: $all
# Required-Stop: $all
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Starts elasticsearch
# Description: Starts elasticsearch using start-stop-daemon
### END INIT INFO
#!/bin/bash
# Elastic Serarch Start and Stop Script
ES_HOME="/opt/elsearch/elasticsearch"
ES_USER="esearch"
PID=$(ps ax | grep elasticsearch | grep $ES_HOME | grep -v grep | awk '{print $1}')
#echo $PID

Simple Website Crawler

The following gist is an extract of the article Building a simple crawler. It allows crawling from a URL and for a given number of bounce.

Basic Usage

from crawler import Crawler
crawler = Crawler()
crawler.crawl('http://techcrunch.com/')

displays the urls

@uolter
uolter / venv_setup.sh
Created September 22, 2014 13:37
venv_and_pip_centos.sh
curl https://raw.githubusercontent.com/pypa/pip/master/contrib/get-pip.py > get-pip.py;
python get-pip.py;
rm -f get-pip.py;
# change directory here. Go in your project home dir.
# cd /opt/uuid_resolver/;
pip install virtualenv;
virtualenv venv;
# activate the virtualenv
source venv/bin/activate
# change here your requirements.txt location
@uolter
uolter / pip_update_all
Created December 2, 2014 09:42
pip update all packages
pip freeze --local | grep -v '^\-e' | cut -d = -f 1 | xargs pip install -U
@uolter
uolter / tree.py
Created December 22, 2014 10:24
Tree Data Structure for Manager Employee Hierarchy
# -*- coding: utf-8 -*-
import unittest
index = {}
class tree(object):
@uolter
uolter / quicksort.py
Created December 29, 2014 10:27
Simple Quck Sort example with python
#!/usr/bin/env
# -*- coding: utf-8 -*-
import unittest
""" Quicksort implementation """
def quicksort(arr):
""" Quicksort a list