leetschau’s gists

leetschau / install redmine with puppet

Created February 11, 2014 03:15

install redmine with puppet

	Today I try to install Redmine server with Puppet on a vagrant managed virtual machine. I use a published module on puppet forge:
	http://forge.puppetlabs.com/johanek/redmine
	And the following is my operations:

	1. Initialize VM:

	chad@typcserver ~/docs/vagrant-prj $ vagrant --version
	Vagrant 1.4.3
	chad@typcserver ~/docs/vagrant-prj $ vagrant init
	chad@typcserver ~/docs/vagrant-prj $ cat Vagrantfile

leetschau / gist:42347eaf142c780b0fa0

Created November 27, 2014 02:45

Spark record delimiter definition

	When I run above codes in spark-shell, I got the following errors:

	scala> val job = new Job(sc.hadoopConfiguration)
	warning: there were 1 deprecation warning(s); re-run with -deprecation for details
	java.lang.IllegalStateException: Job in state DEFINE instead of RUNNING
	at org.apache.hadoop.mapreduce.Job.ensureState(Job.java:283)
	at org.apache.hadoop.mapreduce.Job.toString(Job.java:452)
	at scala.runtime.ScalaRunTime$.scala$runtime$ScalaRunTime$$inner$1(ScalaRunTime.scala:324)
	at scala.runtime.ScalaRunTime$.stringOf(ScalaRunTime.scala:329)
	at scala.runtime.ScalaRunTime$.replStringOf(ScalaRunTime.scala:337)

leetschau / progress-button.html

Created May 20, 2016 05:19

Progress Button Group

<html><br><head><br><link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css" integrity="sha384-1q8mTJOASx8j1Au+a5WDVnPi2lkFfwwEAa8hDDdjZlpLegxhjVME1fgjWPGmkzs7" crossorigin="anonymous"><br><br><style><br>  .progress-group {<br>    margin: 50px;<br>    width: 50%;<br>  }<br>  .progress-button {<br>    position: relative;<br>  }<br>  .progress-button::before {<br>    position: absolute;<br>    border-width: 17px;<br>    border-style: solid;<br>    content: "";<br>    right: -34px; <br>    top: -1px;<br>    border-color: transparent transparent transparent #ccc;<br>    z-index: 50;<br>  }<br>  .progress-button::after {<br>    position: absolute;<br>    border-width: 16px;<br>    border-style: solid;<br>    content: "";<br>    right: -32px; <br>    top: 0;<br>    border-color: transparent transparent transparent white;<br>    z-index: 50;<br>  }<br>  .progress-button.btn-lg::before {<br>    border-width: 23px;<br>    right: -46px; <br>  }<br>  .progress-button.btn-lg::af

leetschau / convertDate.sh

Last active May 28, 2016 12:14

MongoDB导出的json数据中，时间字段前面都有{$date: 2016.04.04}，$date不能被Elasticsearch识别，需要去掉，从{"timeStart: {$date: "xxx"}}变成{"timeStart: "xxx"}

	#!/bin/bash

	sed 's/{\"\$date\":$\".\{10\}T.\{12\}Z\"$}/\1/g' f1000.json > temp.json
	sed -i 25d temp.json
	mongoimport -d test -c strDate --type json --file temp.json

	# sample data:
	"timeStart":{"$date":"1015-09-01T00:00:00.000Z"},"timeEnd":{"$date":"2018-03-02T00:00:00.000Z"},
	"timeStart":{"$date":"2016-09-01T00:00:00.000Z"},"timeEnd":{"$date":"2021-04-0kT00:00:00.000Z"},
	"timeStart":{"$date":"2017-09-01T00:00:00.000Z"},"timeEnd":{"$date":"2021-05-09T00:00:00.000Z"},

leetschau / sorted.py

Created May 28, 2016 13:54

Python sorted 函数，当某项不包含排序依据时，可以将fallback值作为自己的排序依据

	aa = [{'a': 3}, {'a': 2}, {'a': 5}, {'b':8}]
	sorted(aa, key=lambda rec: rec['a'] if 'a' in rec else 1, reverse=True)
	#: [{'a': 5}, {'a': 3}, {'a': 2}, {'b': 8}]
	sorted(aa, key=lambda rec: rec['a'] if 'a' in rec else 6, reverse=True)
	#: [{'b': 8}, {'a': 5}, {'a': 3}, {'a': 2}]

leetschau / cpColl.sh

Created May 28, 2016 14:06

Copy collection between MongoDB servers

	#!/usr/bin/python

	import sys, re
	from os import system

	usage = """
	Usage:
	Full style: ./cpColl sUser:sPwd@sHost:sPort/sDb-sColl dUser:dPwd@dHost:dPort/dDb-dColl
	Abbr for source on localhost: ./cpColl sDb-sColl dUser:dPwd@dHost:dPort/dDb-dColl
	Abbr for destination on localhost: ./cpColl sUser:sPwd@sHost:sPort/sDb-sColl dDb-dColl

leetschau / multi-dimension-correlation-matrix.py

Last active July 7, 2016 03:26

在Anaconda环境下运行，详细分析见"Data Science from Scratch"第10章读书笔记

	import math
	import random
	import numpy as np
	import matplotlib.pyplot as plt


	def normal_cdf(x, mu=0, sigma=1):
	return (1 + math.erf((x - mu) / math.sqrt(2) / sigma)) / 2

leetschau / arguments_parser.py

Last active September 25, 2016 00:29

	import argparse

	parser = argparse.ArgumentParser(
	description='Sync docs in MongoDB to Elasticsearch server')
	parser.add_argument('inputfile_name',
	help='the json file contains data to synced from MongoDB')
	parser.add_argument('-s', '--server-name',
	help='the IP address of Elasticsearch server')
	parser.add_argument('-i', '--index', help='the index name of ES')
	parser.add_argument('-t', '--type-name', help='the type name of the index')

leetschau / dsnote2evernote.py

Created September 17, 2016 10:06

将dsnote里面的笔记通过`geeknote create`命令同步到Evernote上

	import os
	import subprocess

	basedir = '/home/leo/.donno/repo'
	notebook = 'dsnote'

	err_coll = []
	for fn in os.listdir(basedir):
	print('upload %s' % fn)
	fullname = os.path.join(basedir, fn)

leetschau / docker-show-repo-tags.sh

Created September 26, 2016 08:45

list all tags of an image in docker hub

	#!/bin/sh
	# From: http://stackoverflow.com/a/34054903/701420
	# Simple script that will display docker repository tags.
	#
	# Usage:
	# $ docker-show-repo-tags.sh ubuntu centos
	for Repo in $* ; do
	curl -s -S "https://registry.hub.docker.com/v2/repositories/library/$Repo/tags/" \| \
	sed -e 's/,/,\n/g' -e 's/\[/\[\n/g' \| \
	grep '"name"' \| \

	#!/bin/bash

	sed 's/{\"\$date\":\(\".\{10\}T.\{12\}Z\"\)}/\1/g' f1000.json > temp.json
	sed -i 25d temp.json
	mongoimport -d test -c strDate --type json --file temp.json

	# sample data:
	"timeStart":{"$date":"1015-09-01T00:00:00.000Z"},"timeEnd":{"$date":"2018-03-02T00:00:00.000Z"},
	"timeStart":{"$date":"2016-09-01T00:00:00.000Z"},"timeEnd":{"$date":"2021-04-0kT00:00:00.000Z"},
	"timeStart":{"$date":"2017-09-01T00:00:00.000Z"},"timeEnd":{"$date":"2021-05-09T00:00:00.000Z"},

Leo leetschau