Skip to content

Instantly share code, notes, and snippets.

View hungpk's full-sized avatar
😁
Available

Kent Pham hungpk

😁
Available
  • Ho Chi Minh City, Vietnam
View GitHub Profile
@hungpk
hungpk / pgcluu-s3.sh
Last active August 29, 2015 14:07
Collect PostgreSQL stat using pgcluu_collectd and put it to S3
#!/bin/sh
HOST=$1
BASE_DIR=$2
S3BUCKET=<s3-bucket>
NOW=`date +"%Y-%m-%d"`
PREFIX=pgcluu
DATA_DIR=$BASE_DIR/$PREFIX-$NOW
#kill current process
pgcluu_collectd -k -f $BASE_DIR/current/pg_cluu_collectd.pid

Ideal Postgres environment

Documentation

  • Documented replication topology
  • Documented network topology
  • Documented interface topology - including users, passwords, connection estimates, load balancers, connection proxies
  • Documented procedure, schedule for failover and testing
  • Documented procedure, schedule for disaster recovery and testing

WAL-E needs to be installed on all machines, masters and slaves.

How to install WAL-E

Only one machine, the master, writes WAL segments via continuous archiving. The configuration for the master postgresql.conf is:

archive_mode = on
archive_command = 'envdir /etc/wal-e.d/env wal-e wal-push %p'
archive_timeout = 60

Install ImageMagick for image conversion:

brew install imagemagick

Install tesseract for OCR:

brew install tesseract --all-languages

Or install without --all-languages and install them manually as needed.

hostssl replication replicator 5.6.7.8 md5

WAL-E needs to be installed on all machines, masters and slaves.

How to install WAL-E

Only one machine, the master, writes WAL segments via continuous archiving. The configuration for the master postgresql.conf is:

archive_mode = on
archive_command = 'envdir /etc/wal-e.d/env wal-e wal-push %p'
archive_timeout = 60

Types of index scans

Indexes

Sequential Scan:

  • Read every row in the table
  • No reading of index. Reading from indexes is also expensive.
#!/bin/bash
#Force file syncronization and lock writes
mongo admin --eval "printjson(db.fsyncLock())"
MONGODUMP_PATH="/usr/bin/mongodump"
MONGO_HOST="prod.example.com"
MONGO_PORT="27017"
MONGO_DATABASE="dbname"
# run with: god -c /path/to/config.god [add -D if you want to not-deamonize god]
# This is the actual config file used to keep the delayed_job running
APPLICATION_ROOT = "/var/www/application"
RAILS_ENV = "production"
God.watch do |w|
w.name = "delayed_job_production"
w.interval = 15.seconds
w.start = "/bin/bash -c 'cd #{APPLICATION_ROOT}/current; /usr/bin/env RAILS_ENV=#{RAILS_ENV} #{APPLICATION_ROOT}/current/script/delayed_job start > /tmp/delay_job.out'"
curl https://raw.github.com/git/git/master/contrib/completion/git-completion.bash -o ~/.git-completion.bash
echo "source ~/.git-completion.bash" >> ~/.bash_profile