Skip to content

Instantly share code, notes, and snippets.

View qrtt1's full-sized avatar
🎯
Focusing

Ching Yi, Chan qrtt1

🎯
Focusing
View GitHub Profile
作者mrforget (mrforget)
看板Soft_Job
標題[請益] 該離職嗎!?
時間Wed Aug 13 21:54:45 2014
請問各位先進,
我在上星期二面試某間資訊業界的老公司並拿到offer,
當初面試結果:
@qrtt1
qrtt1 / fabfile.py
Created August 12, 2014 13:51
My Cache Server KickStart Scripts
from __future__ import with_statement
from fabric.api import run, env, task, abort, local, hosts, parallel, hide, settings, execute, put, sudo
from fabric.network import needs_host
from string import Template
import os.path
env.user ="root"
env.disable_known_hosts = True
default_key_file = os.path.expanduser('~/.ssh/acme_common.pem')
#if !RELEASE
#define logI(tag, fmt, ...) do{__android_log_print(ANDROID_LOG_INFO, tag, fmt, ## __VA_ARGS__);}while(0)
#define logD(tag, fmt, ...) do{__android_log_print(ANDROID_LOG_DEBUG, tag, fmt, ## __VA_ARGS__);}while(0)
#else
#define logI(tag, fmt, ...)
#define logD(tag, fmt, ...)
#endif

自我學習,以目前限有材料規劃的 learning path

這只是工具的用法,別忘了吸收資料科學的知識 yo

hadoop 入門速記

  1. 用 Hadoop MapReduce Framework 寫程式出來跑得動,跟你想的結果一樣
  2. 參考 popcorny 分享的 word count 專案 https://github.com/popcornylu/hadoop-wordcount
  3. Map Reduce 演算法概念
  4. 弄懂 Map Reduce 演算法各資料處理階段對應至 Hadoop MapReduce Framework 上的 Job Template (之後最佳化會用到)
@qrtt1
qrtt1 / s3.policy.json
Created July 23, 2014 00:42
S3 Policy allow anyone to GetObject from some the source IPs
{
"Id": "Policy1406075254752",
"Statement": [
{
"Sid": "Stmt1406075248586",
"Action": [
"s3:GetObject"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::issue.details/*",
@qrtt1
qrtt1 / configure-pat.sh
Last active August 29, 2015 14:03
VPC NAT instance
#!/bin/bash
# Configure the instance to run as a Port Address Translator (PAT) to provide
# Internet connectivity to private instances.
function log { logger -t "vpc" -- $1; }
function die {
[ -n "$1" ] && log "$1"
log "Configuration of PAT failed!"
exit 1
bash-4.1$ kinit hbase/[email protected] -k -t /var/run/cloudera-scm-agent/process/249-hbase-MASTER/hbase.keytab
bash-4.1$ klist
Ticket cache: FILE:/tmp/krb5cc_484
Default principal: hbase/[email protected]
Valid starting Expires Service principal
05/25/14 14:29:08 05/26/14 14:29:08 krbtgt/[email protected]
renew until 05/25/14 14:29:08
bash-4.1$ klist ^C
bash-4.1$ hbase shell /tmp/hbase_test
@qrtt1
qrtt1 / clean_buffer_cache.txt
Last active August 29, 2015 14:01
etu-misc
http://www.commandlinefu.com/commands/view/1026/empty-the-linux-buffer-cache
sync && echo 3 > /proc/sys/vm/drop_caches
[hdfs@node1 ~]$ hdfs namenode
14/05/21 12:37:13 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = node1/54.255.200.6
STARTUP_MSG: args = []
STARTUP_MSG: version = 2.2.0
STARTUP_MSG: classpath = /opt/hadoop/etc/hadoop:/opt/hadoop/share/hadoop/common/lib/commons-logging-1.1.1.jar:/opt/hadoop/share/hadoop/common/lib/jetty-util-6.1.26.jar:/opt/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/opt/hadoop/share/hadoop/common/lib/zookeeper-3.4.5.jar:/opt/hadoop/share/hadoop/common/lib/activation-1.1.jar:/opt/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/opt/hadoop/share/hadoop/common/lib/guava-11.0.2.jar:/opt/hadoop/share/hadoop/common/lib/commons-compress-1.4.1.jar:/opt/hadoop/share/hadoop/common/lib/jettison-1.1.jar:/opt/hadoop/share/hadoop/common/lib/commons-digester-1.8.jar:/opt/hadoop/share/hadoop/common/lib/servlet-api-2.5.jar:/opt/hadoop/share/hadoop/common/lib/stax-api-1.0.1.jar:/opt/hadoop/s
var firstName = "-first-name-";
var lastName = "-last-name-";
var email = "-email-";
var phone = "-phone-";
var company = "-company-";
var zipCode = "-zip-code-";
var country = "-country-";
var newLimit = "-limitation-";