Pavel Savchenko asfaltboy

Vim Cheatsheet

As configured in my dotfiles

Intro Resources

Your problem with Vim is that you don't grok vi (on StackOverflow)
Coming Home to Vim
VimCasts.org
Vim Galore
Vim Adventures!

Log cronjob output to syslog and send errors to sentry

The wonderful [cron-sentry][1] takes care of sending an event for program errors to sentry. To uniformy log everything outputted, we pipe both stderr and stdout to [logger][2] (should be available on most linux distros)

How to use

Download script, add execution perm and move to a directory on PATH

$ wget https://gist.githubusercontent.com/asfaltboy/23a7f963bbfcb48ce510307ec8cb8737/raw/49ebca20004211a16c85a61a77db7d0883ddf31c/log-and-sentry

$ chmod +x log-and-sentry

JQ to filter JSON by value

Syntax: cat <filename> | jq -c '.[] | select( .<key> | contains("<value>"))'

Example: To get json record having _id equal 611

cat my.json | jq -c '.[] | select( ._id | contains(611))'

Remember: if JSON value has no double quotes (eg. for numeric) to do not supply in filter i.e. in contains(611)

Current

I am using the fantastic jq to manipulate a REST API's json, into a csv for upsertting into another database system, via its API. Once in the target system, I'm doing some date math, including rounding time stamps to 30 min intervals to allow me to do group them, for standard deviation calculation. The problem is, there is a lot of data and the database chokes when it has to round every record and then do std dev calculations on each.

Sample.json shows a small sample of what the input data looks like, but actually it's just stdout from the curl command in the shell script.
example-initial.sh shows the important bit of the initial shell script, that uses curl to authenticate against the data location's API, and then use jq to add a couple of columns and export to csv. It works like a charm.
ACME-X1-A.csv is an example of the output CSV file, that's then upsertted into the target db.

Goal

How to recover lost Python source code if it's still resident in-memory

I screwed up using git ("git checkout --" on the wrong file) and managed to delete the code I had just written... but it was still running in a process in a docker container. Here's how I got it back, using https://pypi.python.org/pypi/pyrasite/ and https://pypi.python.org/pypi/uncompyle6

Attach a shell to the docker container

Install GDB (needed by pyrasite)

apt-get update && apt-get install gdb

	{
	"APIGatewayServiceRolePolicy": {
	"Arn": "arn:aws:iam::aws:policy/aws-service-role/APIGatewayServiceRolePolicy",
	"AttachmentCount": 0,
	"CreateDate": "2019-10-22T18:22:01+00:00",
	"DefaultVersionId": "v6",
	"Document": {
	"Statement": [
	{

	# Save this file
	# mac - "~/Library/Application Support/Sublime Text 3/Packages/"
	# linux - "~/.config/sublime-text-3/Packages/"

	# NOTE!
	# ~/.nvm/alias/default must contain full version name.
	# $ nvm alias default v4.3.5

	import os

	### No longer needed as of nginx-1.13.6-1.el7_4.ngx.x86_64.rpm from nginx.org
	### it was compiled against OpenSSL 1.0.2 from CentoOS 7.4 so it supports ALPN (HTTP2 works)

	yum -y groupinstall 'Development Tools'
	yum -y install wget openssl-devel libxml2-devel libxslt-devel gd-devel perl-ExtUtils-Embed GeoIP-devel rpmdevtools

	OPENSSL="openssl-1.0.2l"
	NGINX_VERSION="1.13.5-1"
	NJS_VERSION="1.13.5.0.1.13-1"

	[program:errbot]
	command = /srv/errbot/virtualenv/bin/errbot --config /srv/errbot/config.py
	user = errbot
	stdout_logfile = /var/log/supervisor/errbot.log
	stderr_logfile = NONE
	redirect_stderr = true
	directory = /srv/errbot/
	startsecs = 3
	stopsignal = INT
	environment = LC_ALL="en_US.UTF-8"

	# Based on CentOS7 fork of @smartmadsoft: https://gist.github.com/moneytoo/ab3f34e4fddc2110675952f8280f49c5

	# "6" for CentOS6 or Amazon Linux, "7" for CentOS7
	CENTVER="6"

	OPENSSL="openssl-1.1.0-pre5"
	NGINX="nginx-1.11.0-1"

	yum clean all
	# Install epel packages (required for GeoIP-devel)