Yanhao Yang YanhaoYang

I build service and API.

YanhaoYang / parse_log.rb

Created May 28, 2019 19:29

Parse Heroku logs and index the logs in Elasticsearch

	require 'elasticsearch'
	require 'time'

	client = Elasticsearch::Client.new host: "172.17.0.1", log: true

	mappings = {
	mappings: {
	log: {
	properties: {
	ts: { type: 'date' },

YanhaoYang / crawler.rb

Created May 27, 2019 14:16

Create a crawler with Capybara

	require 'capybara'
	require 'capybara/dsl'
	require 'selenium-webdriver'

	# java -jar selenium-server-standalone-3.14.0.jar
	#
	if ENV["SELENIUM_REMOTE_HOST"]
	Capybara.register_driver :selenium_chrome_remote do \|app\|
	args = [
	"--disable-infobars",

YanhaoYang / run-jet.sh

Created May 25, 2019 08:27

Archive current repository and run Codeship's jet

	#!/bin/bash

	set -x
	set -e

	root_dir=$(pwd)
	tmpdir=$(mktemp -d "${TMPDIR:-/tmp/}$(basename $0).XXXXXXXXXXXX")

	git archive --prefix=my-jet/ HEAD > "$tmpdir/archive.tgz"

YanhaoYang / aws-ftp-role.json

Created May 25, 2019 08:24

aws ftp role

	{
	"Version": "2012-10-17",
	"Statement": [
	{
	"Sid": "AllowListingOfUserFolder",
	"Action": [
	"s3:ListBucket",
	"s3:GetBucketLocation"
	],
	"Effect": "Allow",

YanhaoYang / delete-git-branches.sh

Created April 28, 2019 18:20

-#!/bin/bash
-delete_branches () {
-  branches=($(git branch | sed 's/*//g'))
-  element_count=${#branches[@]}
-  selected_branches=()
-  echo "Select branches:"
-  for br in "${branches[@]}"
-  do
-    echo "Delete $br ? y/(n)"

YanhaoYang / notes-for-command-lines.sh

Last active May 25, 2019 08:29

Some notes for command lines

	# Find all .orig files, and delete them with confirmation
	find . -name ".orig" && read -q "REPLY?Delete all?" && find . -name ".orig" -delete && echo "\nDone."

	# pg_restore with bandwidth control and progress bar
	cat db.dump \| pv -pera -s $(stat -c %s db.dump) -L 2000000 \| pg_restore -d postgres://postgres:a@localhost/db_name --clean --no-owner --verbose -Fc

	# check pg readiness
	for i in {1..60}; do echo "Waiting for pg ..."; pg_isready -d $PG_URL && break \|\| sleep 1; done

	# http://www.grymoire.com/Unix/Awk.html

YanhaoYang / notes-for-vim.vim

Last active April 13, 2019 09:41

Some notes for vim

	" Copy from Vim to clipboard
	vmap <leader>x :w! ~/.vbuf<cr>:!cat ~/.vbuf \\| xclip -in -sel clip<cr><cr>

	" Strip rspec outputs
	:%s/rspec $.\{-}$ #.\+/\1 \\/gc

YanhaoYang / puma.rb

Created October 19, 2018 18:41

Print the call stacks in a multithread application, say puma, to know which threads are blocked

	before_fork do
	trap 'TTIN' do
	Thread.list.each do \|thread\|
	puts "\nThread TID-#{thread.object_id.to_s(36)}"
	puts thread.backtrace.join("\n")
	puts "\n"
	end
	end
	end

YanhaoYang / queries.sql

Created July 26, 2018 12:29

Some useful queries

	SELECT nspname \|\| '.' \|\| relname AS "relation",
	pg_size_pretty(pg_relation_size(C.oid)) AS "size"
	FROM pg_class C
	LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
	WHERE nspname NOT IN ('pg_catalog', 'information_schema')
	ORDER BY pg_relation_size(C.oid) DESC
	LIMIT 200 -- bigtables;

	SELECT
	pg_stat_activity.pid,

YanhaoYang / docker-compose.yml

Created July 17, 2018 11:06

Docker compose with ssh-agent