3dd13’s gists

3dd13 / batch_concurrent_requests_by_typhoeus.rb

Created April 22, 2010 09:00

example of using Typhoeus


	def main
	# urls is a hash containing url to retrieve statuses of items
	get_csv(urls, items) { \|response, item\|
	item_results = CSV.parse_line(response.body)
	item.setStatus(item_results[0])
	}
	end

	def process_response(urls, items)

3dd13 / ftp_client.py

Created May 19, 2010 16:43

VXRL - Defcon 2010 preparation

	import ftplib
	from ftplib import FTP

	buffer = '\x41' * 400

	try:
	ftp = FTP('127.0.0.1')
	ftp.login(user, password)
	ftp.transfercmd("command" + buffer)
	print 'xxx'

3dd13 / download_open_rice_contact.rb

Created October 19, 2010 08:36

OpenRice.com is a famous restaurant online directory in Hong Kong. Here is a ruby script to grep all the contact information of all shops. at this moment of time, there are around 26K records on the website. this program took ~ 15mins to finish. and

	require 'rubygems'
	require 'mechanize'
	require 'fastercsv'

	def get_all_option_values(page, attr_name)
	page.search("[name=#{attr_name}]").search("option").map do \|opt\|
	option_value = opt.attribute("value").value
	[option_value, opt.text.strip.gsub(/\302\240\302\240/, '')] if option_value && option_value.length > 0
	end.compact
	end

3dd13 / css_xpath_selector_benchmarking.rb

Created November 5, 2010 04:00

just curious about CSS and XPath selector performance. did some benchmarking

	require 'rubygems'
	require 'mechanize'
	require 'benchmark'

	agent = Mechanize.new
	# however, this fetch already spend around 2 seconds. one http request response.
	page = agent.get('http://www.ufood.com.hk/search/search.action?name=&distIds=4,10,11,3,5,8,6,7,2,1,9,12,13,14,15,18,16,17,19,20,21,22,23,24,25,26,33,34,27,35,32,30,29,28,31&ftIds=')

	Benchmark.bm do \|b\|
	# search by tag name and class 10.470000 0.040000 10.510000 ( 11.149709)

3dd13 / 1_controller_test.rb

Created January 7, 2011 07:04

sharing on functional test refactoring with code block. cleaner and more readable.

	test "not allow to update state after account reconciliation" do
	# get fixture
	account = accounts(:fresh)

	# submit and expect no errors
	post :waive_fee, :id => account.id
	assert_nil flash[:error]

	post :add_fee, :id => account.id
	assert_nil flash[:error]

3dd13 / generate_ssh_keys.rb

Created January 20, 2011 16:59

chef recipe to generate ssh key for a user

	define :generate_ssh_keys, :user_account => nil do
	username = params[:user_account]

	raise ":user_account should be provided." if username.nil?

	Chef::Log.debug("generate ssh skys for #{username}.")

	execute "generate ssh skys for #{username}." do
	user username
	creates "/home/#{username}/.ssh/id_rsa.pub"

3dd13 / stock_daily_price.rb

Created January 21, 2011 02:39

Download stock price data from yahoo and insert them into database

	require 'rubygems'
	require 'active_record'
	require 'yaml'

	dbconfig = YAML::load(File.new(File.join(File.dirname(__FILE__), "../../config/database.yml")))
	ActiveRecord::Base.establish_connection(dbconfig["development"])

	class StockDailyPrice < ActiveRecord::Base
	def initialize(stock_values, stock_no = "")
	super()

3dd13 / log_rotation.rb

Created January 21, 2011 02:46

Log Rotation methods provided by Ruby Core http://ruby-doc.org/ruby-1.9/classes/Logger.html

	def logger
	@logger \|\|= Logger.new(File.new("data_loader.log"),3,510241024)
	end

3dd13 / clear_log.rb

Created January 27, 2011 06:34

Adding log purging job using chef

	cron "housekeep application log files" do
	user "housekeeper"
	# mailto "[email protected]"

	# 00:01 every night
	hour "0"
	minute "1"

	# clean up logs older than 30 days
	command "/find ~/log/application-*.log.tar.gz -mtime +30 -exec rm {} \;"

3dd13 / row_is_cut_in_the_middle_solution.rb

Created February 18, 2011 07:49

overflow / page break examples of Prawn Pdf generation.

	# gem "prawn", "0.8.4"
	# Sometime there is no enough space for the last table row when it reaches the end of page

	Prawn::Document.generate("text_group_overflow_question.pdf") do \|pdf\|
	add_page_break_if_overflow(pdf) do \|pdf\|
	# generating table here
	# ...
	end
	end

3dd13 3dd13