Skip to content

Instantly share code, notes, and snippets.

View muxueqz's full-sized avatar

muxueqz muxueqz

View GitHub Profile
worddict = set()
with open ('words.txt') as fh:
lines = fh.readlines()
for element in lines:
words = element.rstrip()
worddict.add(words)
print '{0} words in spelling words'.format(len(worddict))
with open ('3-2_example.txt') as ex:
@muxueqz
muxueqz / log
Created April 13, 2015 10:39
日志系统架构
#当前架构
heka(日志收集器) -> ElasticSearch(分布式搜索引擎,相当于数据库) -> Kibana(界面展现)
##Heka
- 日志收集服务
- Mozilla出品
- Golang编写
- 轻量却功能强大
- 灵活易用
- Go/Lua扩展
#!/usr/bin/env python
def curry(x):
u = bigObj(x)
def f1(y);
def f(a, b):
pass
return f(u, 10)
def curry(x)
u = bigObj(x)
def f1(y);
def f(a, b):
pass
return f(u, 10)
curr(10)(1) # 局部变量 u 会被创建
curr(10)(2) # 这是 u 会重新被创建,还是使用上一个语句 u 的缓存
@muxueqz
muxueqz / Dpark Fetch failed for shuffle
Created May 28, 2014 06:41
Dpark Fetch failed for shuffle
2014-05-26 00:18:22,040 [INFO] [broadcast] broadcast e96a2f61-fdd6-45d9-9be7-b50931e3fc33: 291 bytes in 1 blocks
2014-05-26 00:18:22,048 [INFO] [broadcast] broadcast 14e902be-8d9b-4a47-809e-589e7a1b6a34: 5 bytes in 1 blocks
2014-05-26 00:18:22,049 [INFO] [broadcast] broadcast 5d791168-2b67-428d-95ce-b7b83981a890: 3218 bytes in 1 blocks
2014-05-26 00:18:22,087 [INFO] [scheduler] Got job 1 with 693 tasks: <FlatMappedRDD <UnionRDD 1 <UnionRDD 24 <GZipFileRDD /mfs/logs/out-2014052500.log.gz> ...> ...>>
2014-05-26 00:19:41,069 [INFO] [job ] Job 1 finished in 79.0s: min=1.4s, avg=5.4s, max=9.2s, maxtry=1
2014-05-26 00:19:41,128 [INFO] [scheduler] Got job 2 with 441 tasks: <MappedRDD <FilteredRDD <MappedRDD <ShuffledRDD <FlatMappedRDD <UnionRDD 1 <UnionRDD 24 <GZipFileRDD /mfs/logs/out-2014052500.log.gz> ...> ...>>>>>>
2014-05-26 00:19:41,132 [INFO] [scheduler] Got job 3 with 441 tasks: <MappedRDD <FilteredRDD <MappedRDD <Shuffl
@muxueqz
muxueqz / ejabberd.cfg
Created October 8, 2013 06:13
Archipel ejabberd.cfg
%%%
%%% ejabberd configuration file
%%%
%%% The parameters used in this configuration file are explained in more detail
%%% in the ejabberd Installation and Operation Guide.
%%% Please consult the Guide in case of doubts, it is included in
%%% your copy of ejabberd, and is also available online at
%%% http://www.process-one.net/en/ejabberd/docs/
#!/usr/bin/env python
import sys
print(sys.stdin.read())
#!/usr/bin/env luajit
-- Copyright (C) 2012 by Yichun Zhang (agentzh)
local ffi = require "ffi"
local assert = assert
module(...)
_VERSION = '0.01'
@muxueqz
muxueqz / company wifi eap
Created June 20, 2013 10:16
company wifi eap
name = Uni
author = Me
version = 1
require identity *Identity password *Password
-----
ctrl_interface=/var/run/wpa_supplicant
network={
ssid="$_ESSID"
scan_ssid=$_SCAN
proto=RSN
#!/usr/bin/env python2
#-*- coding: utf-8 -*-
"使用mesos多机的话,在reduceByKey时None的key没有合并,单机无此问题"
import dpark
def xmap_none(line):
for i in range(1, 3):
key = (str(i), None)
yield (key, ({line}, 1))