Skip to content

Instantly share code, notes, and snippets.

View shiumachi's full-sized avatar

Sho Shimauchi shiumachi

View GitHub Profile
@shiumachi
shiumachi / gist:8373453
Last active January 2, 2016 22:59
配列から簡単なヒストグラムを作成する。
def hist(arr, meter=20):
""" print histogram.
Arguments:
arr: dataset. contents must be numeric
meter (default to 20)
"""
for n in arr:
print("{0:>4}: {1}".format(n, "*" * int(round(float(n) / (float(max(arr)) / meter)))))
@shiumachi
shiumachi / gist:8387309
Created January 12, 2014 16:56
シンプルなWSGIサーバ
def application(environ, start_response):
start_response('200 OK', [('Content-type', 'text/plain')])
return "Hello, world!"
from wsgiref import simple_server
if __name__=='__main__':
server = simple_server.make_server('', 8080, application)
server.serve_forever()
@shiumachi
shiumachi / gist:8402874
Created January 13, 2014 16:07
requests + BeautifulSoup で github の h1 タグをとってくる
from BeautifulSoup import BeautifulSoup
import requests
r = requests.get("https://github.com/")
soup = BeautifulSoup(r.text)
for i in soup.findAll('h1'):
print(i.text)
@shiumachi
shiumachi / gist:8421275
Created January 14, 2014 16:37
NameNode の jmx から used heap size のメトリクスを取得する
import requests
def get_nn_used_heap(nn_jmx_url):
""" return NN used heap (bytes)
Argument:
nn_jmx_url (required)
Return Value:
@shiumachi
shiumachi / gist:8438265
Created January 15, 2014 15:28
ls -l の結果をパースして、辞書にして返す。
def parse_ls(ls_line):
""" parse a line of ls -l result.
Argument:
ls_line (one line of ls -l)
Return:
dict of file / dir information.
permission, filenum, user, group, size, month, day, time, name
"""
@shiumachi
shiumachi / add_yeardayhour_to_log
Created January 16, 2014 16:35
Hadoopのログの先頭にYYYYMMhhの時間タグを付与する。 convert_to_datetime() については別 gist 参照 https://gist.github.com/shiumachi/8269821
from convert_to_datetime import convert_to_datetime
def add_yeardayhour_to_log(line):
""" add YYMMDDhh to the beginning of the log.
Argument:
line (hadoop log line)
"""
arr = line.rstrip().split()
date_string = ' '.join(arr[0:2])
@shiumachi
shiumachi / gist:8477756
Created January 17, 2014 17:36
名前つきのアイテム集合を作成する
# -*- coding: utf-8 -*-
# 参考: Python クックブック第2版 p.183
class Region(object):
def __init__(self, **kwds):
self.__dict__.update(kwds)
@shiumachi
shiumachi / gist:8500700
Created January 19, 2014 05:11
確率が重み付けされたアイテムからランダムに1つ取り出す
# -*- coding: utf-8 -*-
# 参考: Python クックブック第2版 p.188
import random
def random_pick(item_list, probabilities):
""" return a random item in a list, which has different probability.
@shiumachi
shiumachi / get_href_list.py
Last active January 3, 2016 20:49
指定したURLのrequestオブジェクトからhrefのリストを取得する (注: BS3 のコード。BS4 のコードを見たい場合はこちらを参照 https://gist.github.com/shiumachi/8633275 )
# -*- coding: utf-8 -*-
# 参考: http://kondou.com/BS4/
# note: This code uses BeautifulSoup3 which is deprecated.
# If you need code sample of BS, please see https://gist.github.com/shiumachi/8633275
from BeautifulSoup import BeautifulSoup
import requests
@shiumachi
shiumachi / print_two_version.py
Created January 20, 2014 15:10
Pythonのバージョンを判断して処理内容を変える
import sys
if __name__ == '__main__':
# <2.6 doesn't support named value like sys.version_info_major
if sys.version_info[0] == 2 and sys.version_info[1] < 6:
print "Python version: %d.%d" % (sys.version_info[0], sys.version_info[1])
else:
print("Python version: {0}.{1}".format(sys.version_info[0], sys.version_info[1]))