Hartley Brody hartleybrody

I do this infrequently enought that I always forget how to do it. Note that I've finally decided to move away from virtualenvwrapper (workon, mkvirtualenv, etc) since it didn't play nicely with the python versions set by pyenv.

Install the tools (pyenv & pyenv-virtualenv)

Only necessary if this is a new laptop

brew install pyenv
brew install pyenv-virtualenv

kotlin cheatsheet

string interpolation

val foo = 1
var bar = "one is $foo"
var baz = "two is ${foo + 1}"

Unit is a placeholder return value for functions that don't actually return a meaninfgul value. It is the default return type for any function if one is not specified explicitly. Nothing is a placeholder return value for functions that should never actually return (ie they throw an exception or have infinite loop). Good SO discussion on differences.

Pagination Details

Everyone loves to say "you're probably doing pagination wrong" -- in terms of writing efficient SQL queries.

Here is the tl;dr:

make sure you are using ORDER BY in your query, otherwise subsequent queries with a LIMIT and OFFSET could either skip rows or duplicate rows across pages
when going to next page, don't use LIMIT .. OFFSET since the database still has to fetch all rows, then order them, then move to the correct page
instead using something like WHERE id > {max_id_from_prev_page} and (assuming there's an index on that column) the database can jump directly to the correct spot to read just the number of rows you're displaying on the next page
this helps to keep the query constant time as the size of the data grows and/or as you page deeper into the data

Intro

For when you need to insert or update millions of rows and don't want to mess around with scripting to batch individual SQL commands, it's worth using postgres's built in, super-fast support for importing and exporting data from the CSV file format.

Note: that we're using the \COPY command and not the COPY command, since it allows us to work with files locally available on our machine and not worry about transferring files to the database server's file system.

Via the psql manual on \COPY:

Performs a frontend (client) copy. This is an operation that runs an SQL COPY command, but instead of the server reading or writing the specified file, psql reads or writes the file and routes the data between the server and the local file system. This means that file accessibility and privileges are those of the local user, not the server, and no SQL superuser privileges are required.

Export data from database t

Postgres is usually run automatically on my laptop. I also have a weekly cronjob that does brew update and brew upgrade which sometimes bumps the version of portgres that I'm running. Newer versions of the postgres server software aren't necessarily compatible with the older version's data directory structure, and so you need to "upgrade" your database to see all of your old data in the upgraded postgres server.

Trying to start the database manually with

pg_ctl -D /usr/local/var/postgres -l /usr/local/var/postgres/server.log start

Which gives an error message and tells me where to look (note that I passed in the log directory when I ran the command)

waiting for server to start.... stopped waiting

pg_ctl: could not start server

	"""
	Example of setting up multiple click commands within one file.
	Run these commands with:

	python /path/to/file.py do-foo --foo 'hello world'

	Note that click converts '_' to '-' in function name

	"""

	# 1. download a copy of prod db to localhost

	# connect to remote redis database and
	# download a snapshot of the entire database
	# to a local file on your filesystem

	# via https://stackoverflow.com/a/61365048/625840
	# docs https://redis.io/docs/manual/cli/#remote-backups-of-rdb-files

	redis-cli -u $PROD_REDIS_URL --rdb dump.rdb

	# copy/import data from heroku postgres to localhost pg database

	# useful for copying admin's work on live site into local database to reproduce errors
	# https://devcenter.heroku.com/articles/heroku-postgres-import-export

	# take heroku pg snapshot and download
	heroku pg:backups:capture
	heroku pg:backups:download

	# load the dump into local postgres database, assuming $DATABASE_URL set locally

	import json
	import simplejson # pip install simplejson
	import pickle
	import cPickle
	import marshal
	import yaml # pip install pyyaml
	import ujson # pip install ujson
	from cStringIO import StringIO
	import bson # pip install bson

	// Use Gists to store code you would like to remember later on
	console.log(window); // log the "window" object to the console