mezzatto’s gists

mezzatto / dia 1

Last active May 16, 2018 21:42

qconsp2018 - https://qconsp.com/schedule/tabular

	performance (netflix) - https://qconsp.com/system/files/keynotes-slides/qcon_sp_keynote_-_martin_spier.pdf
	- metricas impressionantes deles (requests, logs, latencia, instancias etc)
	- arquitetura (caches nos provedores)
	- 3 regioes aws
	- ec2 reserved (se nao tá streaming, usa pra recomendacoes, encoding etc)
	- build: amis ubuntu
	- deploy: 4000 por dia
	- tudo stateless
	- ferramentas (captura de metricas da redhat, flamegraph, flamescope etc)

mezzatto / cleanupES.sh

Created August 30, 2016 20:20

ES Cleanup

	cd /usr/local/elasticsearch/data/elasticsearch_procura_access/nodes/0/indices

	# Remove indices estranhos gerados por requisicoes que nao sao do logstash
	# Isso acontece, por exemplo, com sistema de deteccao de vulnerabilidade que a infra roda
	echo "Removendo indices estranhos..."
	for i in `ls \| grep -v ^logstash \| grep -v ^ck \| grep -v kibana`; do echo -n "Removendo indice" $i ": "; GET -m DELETE http://127.0.0.1:9200/$i; echo ""; done

	# Remove indices antigos caso o disco esteja cheio
	echo "Removendo indices devido a falta de espaco em disco..."
	while [ "`du -sb \| cut -f 1`" -gt "50000000000" ]; do idx=`ls -tar \| grep -E "(logstash\|marvel\|ck)" \| grep -v kibana \| head -1`; echo -n "Removendo indice" $idx ": "; GET -m DELETE http://127.0.0.1:9200/$idx; echo ""; sleep 1; done

mezzatto / gist:1ce2f5b388a21016b19d4c854538f66c

Created August 24, 2016 14:43

Tracking Update

	public class TrackingUpdateRequestProcessor extends UpdateRequestProcessor {
	private final SchemaField sf;
	public static ConcurrentHashMap<BytesRef, Long> changes = new ConcurrentHashMap<>();

	public TrackingUpdateRequestProcessor(SolrQueryRequest req, UpdateRequestProcessor next) {
	super(next);
	sf = req.getCore().getLatestSchema().getUniqueKeyField();
	}

	@Override

mezzatto / Chain

Created August 24, 2016 14:38

Update Chain

	<updateRequestProcessorChain name="trackingchain" default="true">
	<processor class="solr.LogUpdateProcessorFactory" />
	<processor class="solr.DistributedUpdateProcessorFactory" />
	<processor class="com.vivareal.solr.plugin.processor.TrackingUpdateRequestProcessorFactory" />
	<processor class="solr.RunUpdateProcessorFactory" />
	</updateRequestProcessorChain>

mezzatto / gist:8c18c3ea68bb7e8872f21a5a776636ab

Created August 24, 2016 13:53

CPUs

int nthreads = Math.max(1, Runtime.getRuntime().availableProcessors() / 2);

mezzatto / readBody.c

Created August 11, 2016 17:02

Read HTTPD request body

	#include <strings.h>
	#include <sys/types.h>
	#include <unistd.h>
	#include <zlib.h>

	static long int uncompressedBytes(apr_pool_t* p, char** uncompressedData, const void* bytes, size_t length)
	{
	z_stream strm;
	int ret;
	unsigned char out[128 * 1024];

mezzatto / Example output

Created April 8, 2016 20:12

Docket "top"

	CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O
	centauro-prod-1-3-score-8d6e68 0.00% 0 B / 0 B 0.00% 0 B / 0 B 0 B / 0 B
	dragon-03232016-street 5.80% 226.9 MB / 3.95 GB 5.74% 0 B / 0 B 50.63 MB / 16.38 kB
	dragon-chino-prod-1-3-score-03302016-score-8c292e 8.37% 236.3 MB / 3.95 GB 5.98% 0 B / 0 B 49.5 MB / 12.29 kB
	indexer-03232016-street 0.06% 253.7 MB / 3.95 GB 6.42% 574 kB / 152.8 kB 63.71 MB / 12.29 kB
	indexer-03302016-score 0.18% 242.6 MB / 3.95 GB 6.14% 137.3 kB / 151.9 kB 63.74 MB / 12.29 kB
	locations-monitor-prod 0.00% 70.93 MB / 3.95 GB 1.80% 43

mezzatto / config_m1

Created September 18, 2014 17:21

	bind 172.30.104.126
	port 26379
	unixsocket "/var/run/redis_sentinel.sock"
	daemonize yes
	pidfile "/var/run/redis_sentinel.pid"
	logfile "/var/log/redis/redis_sentinel.log"
	loglevel notice
	timeout 240

	sentinel monitor Group_4 172.30.104.126 60004 2

mezzatto / gist:699c38c55354f15cff71

Created September 18, 2014 17:07

	[root@redis-lhv-01t ~]# while : ; do ps fawux \| grep sentinel \| grep -v grep; sleep 1; done
	root 56761 8.1 0.0 139812 8656 ? Ssl 11:53 0:02 redis-sentinel 172.30.104.126:26379
	root 56761 8.2 0.0 139940 8768 ? Ssl 11:53 0:02 redis-sentinel 172.30.104.126:26379
	root 56761 8.2 0.0 140068 8856 ? Ssl 11:53 0:02 redis-sentinel 172.30.104.126:26379
	root 56761 8.3 0.0 140068 8956 ? Ssl 11:53 0:02 redis-sentinel 172.30.104.126:26379
	root 56761 8.3 0.0 140196 9060 ? Ssl 11:53 0:02 redis-sentinel 172.30.104.126:26379
	root 56761 8.3 0.0 140324 9136 ? Ssl 11:53 0:02 redis-sentinel 172.30.104.126:26379
	root 56761 8.0 0.0 140452 9224 ? Ssl 11:53 0:02 redis-sentinel 172.30.104.126:26379
	root 56761 8.1 0.0 140452 9320 ? Ssl 11:53 0:02 redis-sentinel 172.30.104.126:26379
	root 56761 8.0 0.0 140580 9424 ?

mezzatto / gist:ea6f769c6d5c1eb0a55a

Last active August 29, 2015 14:01

sorted set rate limiter

	local ts = tonumber(table.remove(ARGV))
	local weight = tonumber(table.remove(ARGV))
	local limit = tonumber(ARGV[1])

	redis.call('zremrangebyscore', KEYS[1], 0, tonumber(ts) - 60)
	local curr = redis.call('zrangebyscore', KEYS[1], tonumber(ts), tonumber(ts))
	redis.call('zremrangebyscore', KEYS[1], tonumber(ts), tonumber(ts))
	redis.call('zadd', KEYS[1], ts, tonumber(curr[1] or '0') + tonumber(weight))

	local all = redis.call('zrange', KEYS[1], 0, -1)

Daniel Mezzatto mezzatto