Skip to content

Instantly share code, notes, and snippets.

View sjlombardo's full-sized avatar

Stephen Lombardo sjlombardo

View GitHub Profile
Test 1: 1000 INSERTs
CREATE TABLE t1(a INTEGER, b INTEGER, c VARCHAR(100));
INSERT INTO t1 VALUES(1,13153,'thirteen thousand one hundred fifty three');
INSERT INTO t1 VALUES(2,75560,'seventy five thousand five hundred sixty');
... 995 lines omitted
INSERT INTO t1 VALUES(998,66289,'sixty six thousand two hundred eighty nine');
INSERT INTO t1 VALUES(999,24322,'twenty four thousand three hundred twenty two');
INSERT INTO t1 VALUES(1000,94142,'ninety four thousand one hundred forty two');
@sjlombardo
sjlombardo / backup_pg.sh
Created March 9, 2012 19:03
PostgreSQL Daily Continuous Backup Script
#!/bin/bash
backup_dir=/backup/pg_backups
wal_dir=/backup/postgresql
today=$(date +%Y-%m-%d.%H:%M:%S)
yesterday=$(find $backup_dir -maxdepth 1 -type d -name "*" -exec basename {} \; | tail -n +2 | grep -v "lost+found" | sort -rn | head -1)
psql -c "select pg_start_backup('$today', true);"
if [ $? = 0 ]
@sjlombardo
sjlombardo / gist:2068863
Created March 18, 2012 04:19
500x PBKDF2-HMAC-SHA256 vs 4000x PBKDF2-HMAC-SHA1

Assumptions:

  • PBKDF2 with HMAC-SHA1 must execute SHA1 iterations twice to generate a 256 bit key (digest size is < derived key size)
  • Each HMAC-SHA1 operation executes two internal SHA1 operations.
  • Thus, 4000 iterations of PBKDF2 using HMAC-SHA1 requires 16,000 SHA1 operations
  • PBKDF2 with HMAC-SHA256 executes 1 SHA1 operations per iteration to generate a 256 bit key (digest size = derived key size)
  • Each HMAC-SHA256 operation executes two internal SHA256 operations.
  • Thus, 500 iterations of PBKDF2 using HMAC-SHA256 requires 1000 SHA-256 operations (1/16th the number of operations)
  • SHA256 is slower than SHA1, though it's difficult to get exact benchmarks. For ease of caclulations let us assume that on a GPU, SHA1 is twice as fast as SHA256 (potential sources: http://hashcat.net/oclhashcat-lite/, http://www.insidepro.com/eng/egb.shtml)
@sjlombardo
sjlombardo / gist:2245989
Created March 30, 2012 02:40
SQLCipher Backwards Compatibility with HMAC and aes-256-cfb
new-host% git checkout 1.1.x
Switched to branch '1.1.x'
new-host% ./configure --enable-load-extension --enable-tempstore=yes CFLAGS="-DSQLITE_HAS_CODEC -DSQLITE_ENABLE_FTS3" LDFLAGS="-ltcl8.5 -lcrypto"
...
new-host% make
...
new-host% ./sqlite3 11x-aes-256-cfb.db
SQLite version 3.7.9 2011-11-01 00:52:41
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
@sjlombardo
sjlombardo / 3dbspeed-noopt.html
Created May 8, 2012 20:26
SQLCipher 3-way speed comparison
Test 1: 1000 INSERTs
CREATE TABLE t1(a INTEGER, b INTEGER, c VARCHAR(100));
INSERT INTO t1 VALUES(1,13153,'thirteen thousand one hundred fifty three');
INSERT INTO t1 VALUES(2,75560,'seventy five thousand five hundred sixty');
... 995 lines omitted
INSERT INTO t1 VALUES(998,66289,'sixty six thousand two hundred eighty nine');
INSERT INTO t1 VALUES(999,24322,'twenty four thousand three hundred twenty two');
INSERT INTO t1 VALUES(1000,94142,'ninety four thousand one hundred forty two');
Stock SQLite: 1.507
@sjlombardo
sjlombardo / gist:2919023
Created June 12, 2012 17:56
reproducing karin's script in c with prepared statements
/*
generates a large amount of load for stress testing via repeted opens, inserts and deletes
gcc test.c -I../sqlcipher -DSQLITE_HAS_CODEC -l crypto -o test
*/
#include <sqlite3.c>
#include <stdio.h>
#define ERROR(X) {printf("[ERROR]: "); printf X;fflush(stdout);}
@sjlombardo
sjlombardo / gist:4665741
Created January 29, 2013 16:51
libsqlfs logs
root@ubuntu:/home/sjlombardo/Documents/code/libsqlfs# rm /tmp/fsdata*
root@ubuntu:/home/sjlombardo/Documents/code/libsqlfs# ls /tmp/fsdata*
ls: cannot access /tmp/fsdata*: No such file or directory
root@ubuntu:/home/sjlombardo/Documents/code/libsqlfs# ./.libs/fuse_sqlfs -o allow_other /mnt/sqlfs/
root@ubuntu:/home/sjlombardo/Documents/code/libsqlfs# cp /etc/hosts /mnt/sqlfs/
root@ubuntu:/home/sjlombardo/Documents/code/libsqlfs# ls /tmp/fsdata*
/tmp/fsdata /tmp/fsdata-shm /tmp/fsdata-wal
root@ubuntu:/home/sjlombardo/Documents/code/libsqlfs# fusermount -u /mnt/sqlfs
root@ubuntu:/home/sjlombardo/Documents/code/libsqlfs# ls /tmp/fsdata*
/tmp/fsdata
-----BEGIN PGP PUBLIC KEY BLOCK-----
mQINBFNWuzMBEACryX8m38NyForH6BSJ2UASdm0UNBQVXZIgu2zUZDur2n2SSzFS
Bf3mUPfvRVwQWUQYm0hOySMQDson3sDzoy2AIC0ThfYAgeFK1oichhuxecXkHOco
VkZYtlmu6gq4T/728PkMBGvJVUhNtttNc6i3/oXdYGl0Ea8c2BeEq66QCsgZ1BP3
aO0OQmdVFn6sgJXznkeG1x1yXC0c0928IEioYSzlBy7Q63U56KR4+ih9H1CxKREu
qi2p0b/Jz12d3hyFoJgtfP8mAaGmPgNxOQEJCNUdhOT4e3FBworto32rK6su8TYZ
ZJykUm/K99UsfqB9r8N1IwaDoByZCNpEZPRiJEtfds3FOKW2QUU//7JBmDAxNO1G
DsZR9LsfYYq59uBSDMF0bssd7p+QmvtSZ5jB2yMS+93Ng5Oyj4bHz8S4kMWDR3yK
FGoSOZsRaGHjFO3SiA1GX7v3YM1KPjd4/WL2HNm6KviynoENMVzVc1kPs3nekZMi
-----BEGIN PGP PUBLIC KEY BLOCK-----
mQINBFNWu9kBEADG/o6LmS0T+8g4gXfCkCmLGffWjB3RAomCwDCxAbC006hj06ES
6Uv2uik7Uw4kRd2j9NNuO10QZ7vd9ZfRwtL7uWPWkW9XF/PS6lbcz3C15iCiTa5I
bZaGiErOsHbszufVknqPlEWGpuQCxwJ+hb0pQBCCrJ0aZY2FHG4zDunPkSzDGwW1
aHhKrtxIpVecFZoTaApKDHSTD8IUywTmX+s219aEOltHJjO+avmU4UVxiHUwiPSS
sidLvyELBA96PGKdO/Qi4+xstC1wx3o1vThgACgp71T+26HEnwKXjMeaoyz/e1dD
Z2YiXp1rXVUY49xc3kLeDlexuOQM/ryjroUg1GRkQkL31jbrDx1sVKGUncO/hWrb
Cnt6lnr35hIIf76ZsKKJGe+jqiMAeAMxuu9JCwabE3baStH5ZFgAWAO+VpVey+bp
yJRQT6hbpDPS786fb+EZLBp+I+zxDj21G059JFnZmC26EPTV7/8JEIDtgav3/J5p
@sjlombardo
sjlombardo / transition-statement.txt.asc
Created April 23, 2014 20:16
Stephen Lombardo OpenPGP Key Transition.txt
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
Date: April 23rd, 2014
For a number of reasons, I've recently set up a new OpenPGP key,
and will be transitioning away from my old one.
The old key will continue to be valid for some time, but I prefer all
future correspondence to come to the new one. I would also like this