Skip to content

Instantly share code, notes, and snippets.

@jgillman
jgillman / restore.sh
Last active June 9, 2024 00:46
pg_restore a local db dump into Docker
# Assumes the database container is named 'db'
DOCKER_DB_NAME="$(docker-compose ps -q db)"
DB_HOSTNAME=db
DB_USER=postgres
LOCAL_DUMP_PATH="path/to/local.dump"
docker-compose up -d db
docker exec -i "${DOCKER_DB_NAME}" pg_restore -C --clean --no-acl --no-owner -U "${DB_USER}" -d "${DB_HOSTNAME}" < "${LOCAL_DUMP_PATH}"
docker-compose stop db
@bryanrite
bryanrite / safe.md
Last active October 24, 2024 16:27
Safe Postgres Operations on High Volume Tables

Originally taken from: Braintree Article and expanded on by me.

Safe

  • Add a new column
  • Drop a column
  • Rename a column
  • Add an index concurrently (Example), Note: it will still take a long time to run the migration, but it won't write-lock the table.
  • Drop a constraint (for example, non-nullable)
  • Add a default value to an existing column
@benallard
benallard / xml_split.py
Last active August 28, 2024 23:00
Small python script to split huge XML files into parts. It takes one or two parameters. The first is always the huge XML file, and the second the size of the wished chunks in Kb (default to 1Mb) (0 spilt wherever possible) The generated files are called like the original one with an index between the filename and the extension like that: bigxml.…
#!/usr/bin/env python
import os
import xml.parsers.expat
from xml.sax.saxutils import escape
from optparse import OptionParser
from math import log10
# How much data we process at a time
@campoy
campoy / letitfail.go
Last active January 6, 2024 07:26
This example shows how to have a set of goroutines running concurrently and processing requests. Panics from goroutines are recovered and the worker is restarted. You can download it and run it directly using `go run letitfail.go`
package main
import (
"bufio"
"fmt"
"os"
"time"
)
const numWorkers = 3