Skip to content

Instantly share code, notes, and snippets.

View solos's full-sized avatar
🎶
Listening music

solos

🎶
Listening music
View GitHub Profile
@solos
solos / crawler.py
Created October 10, 2013 00:55 — forked from jmoiron/crawler.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""Simple async crawler/callback queue based on gevent."""
import traceback
import logging
import httplib2
import gevent

Latency numbers every programmer should know

L1 cache reference ......................... 0.5 ns
Branch mispredict ............................ 5 ns
L2 cache reference ........................... 7 ns
Mutex lock/unlock ........................... 25 ns
Main memory reference ...................... 100 ns             
Compress 1K bytes with Zippy ............. 3,000 ns  =   3 µs
Send 2K bytes over 1 Gbps network ....... 20,000 ns  =  20 µs
SSD random read ........................ 150,000 ns  = 150 µs

Read 1 MB sequentially from memory ..... 250,000 ns = 250 µs

@solos
solos / urlnorm.py
Created September 8, 2013 09:23 — forked from mnot/urlnorm.py
#!/usr/bin/env python
"""
urlnorm.py - URL normalisation routines
urlnorm normalises a URL by;
* lowercasing the scheme and hostname
* taking out default port if present (e.g., http://www.foo.com:80/)
* collapsing the path (./, ../, etc)
* removing the last character in the hostname if it is '.'
<!doctype html>
<html>
<head>
<title>粘贴传图Ctrl + V</title>
<script src="jquery-1.6.1.min.js" charset="utf-8"></script>
</head>
<body>
<p id="notice"></p>
<script language="javascript">
window.onload=function(){
// requires
var utils = require('utils');
var casper = require('casper').create()
var casper = require('casper').create({
verbose: true,
logLevel: "debug"
});
// setup globals
var email = casper.cli.options['email'] || 'REPLACE THIS EMAIL';
from selenium import webdriver
from selenium.common.exceptions import TimeoutException
import selenium.webdriver.support.wait
selenium.webdriver.support.wait.POLL_FREQUENCY = 0.05
import re
import random
import collections
class AdwordsAutomater(object):
from gzip import GzipFile
import zlib
from StringIO import StringIO
import urllib2
class GzipHandler(urllib2.BaseHandler):
"""A handler to add gzip capabilities to urllib2 requests """
# add headers to requests
def http_request(self, req):
@solos
solos / ping.py
Created June 8, 2013 05:02 — forked from pklaus/ping.py
#!/usr/bin/env python
"""
A pure python ping implementation using raw socket.
Note that ICMP messages can only be sent from processes running as root.
Derived from ping.c distributed in Linux's netkit. That code is
body {
font-family: Helvetica, arial, sans-serif;
font-size: 14px;
line-height: 1.6;
padding-top: 10px;
padding-bottom: 10px;
background-color: white;
padding: 30px; }
body > *:first-child {
"""
test commit creation with pygit2
To see the result:
rm -rf foo && python test_cc.py && cd foo/ && git log --graph --oneline --date-order --decorate --color --all && git status && cd ..
"""
import os
import sys
import pygit2