Josh Devins joshdevins

Senior Principal Engineer, ML Lead for Search @elastic

joshdevins / gist:d784300ed92350fa60cfb37be55db88e

Created January 7, 2025 15:11

keybase.md

	### Keybase proof

	I hereby claim:

	* I am joshdevins on github.
	* I am joshdevins (https://keybase.io/joshdevins) on keybase.
	* I have a public key ASCVfj5vd1fwp8gNrSlb6Gp-YctfbXzjKU8LxQt9U3mLrAo

	To claim this, I am signing this object:

joshdevins / tcpdump.bash

Last active November 12, 2019 12:11

Capture Elasticsearch queries

	# everything
	sudo tcpdump -i lo0 -A -n -s 0 'tcp dst port 9200 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'

joshdevins / extract.bash

Last active October 16, 2019 14:22

Extract structured query source from a Elasticsearch slow query log for a given an index pattern

jq -r .message | sed -n '/\[\(\INDEX_PATTERN\)\]\[.*\] .*/p' | sed -n 's/.* source\[\(.*\)\]\, id.*/\1/p' | jq -C .

joshdevins / sparkTuning.scala

Created March 27, 2017 08:40

Spark 1.4.1 shell script to tune memory settings

	/**
	* Determines the compute and memory layouts given a Spark 1.4.x configuration.
	*
	* This should be run by applying all of the configuration except for the
	* executor settings (as this would waste resources). E.g.:
	*
	spark-shell \
	--master "local" \
	--deploy-mode "client" \
	--driver-memory 1G \