Skip to content

Instantly share code, notes, and snippets.

View matthew-n's full-sized avatar

Matt N. matthew-n

  • Houston, Tx
View GitHub Profile
@matthew-n
matthew-n / blkdev_max.fio
Last active December 28, 2020 21:59
Linux syth postgres io workload
[global]
ioengine=libaio
direct=1
invalidate=1
verify=0
ramp_time=2s
randrepeat=0
time_based
runtime=5m
filesize=2G
@matthew-n
matthew-n / db_disk_test.sh
Created May 2, 2018 16:23
disk benchmark for database with fio
#!/bin/bash
set -x
set -e
sec_list="seq-read seq-write random-read random-write active-db"
rw_rate_list="75 80 85 90"
io_thread_list="1 4 8 16 24"
for class in $sec_list; do
for ratio in $rw_rate_list; do
@matthew-n
matthew-n / pg_changefee.md
Created May 2, 2018 16:20
idea for implementing changefeeds in postgresql

Overview

table -> after trigger (U/I/D) -> eval quals -> write out (old_ctid, new_ctid, table_name, timestamp) to stg client process collects from stg table only one table_name selecting old_ctid and new_ctid for each

#!/bin/bash
#set -x
#set -e
page_size=`getconf PAGE_SIZE`
phys_pages=`getconf _PHYS_PAGES`
phys_memory_kb=$[ $phys_pages * $page_size / 1024 ]
kb_pct=$[ 1024 / 100]
@matthew-n
matthew-n / featCollection.sql
Last active August 30, 2017 02:51
explode a feature collection into a table
/**
* return: a table of all valid polygons found in the featue collection along with their properties
* error:
* - will not stop execution geometry will be set to empty (should be null?)
* - for all indexes with an error an object with the key `error` and an array
* of deatails will be appended
* ```
* {"error": [ {"reason":'cross over', "locaiton":'<WKT>'}
* ,{"reason:": "invalid type"}]}
* ```
@matthew-n
matthew-n / function-template.sql
Last active December 29, 2020 17:17
PosgreSQL function error handler that displays comments
CREATE OR REPLACE FUNCTION dummy_function( col_pk int, col_b my_domain, col_fk text)
AS
$BODY$
declare
v_msg text;
v_constraint text;
BEGIN
INSERT INTO some_table VALUES($1,$2,$3);
RETURN TRUE;
@matthew-n
matthew-n / plpgsql-notes.md
Last active August 5, 2017 03:43
things not so obvious about plpgsql

End of §41.5.3 - INTO key word

When selecting into a variable it will emit under-select, and over selection if you use strict, as exception types.

SELECT _col_ into strict _v_something_
EXCEPTION
  WHEN NO_DATA_FOUND THEN
    RAISE ...
  WHEN TOO_MANY_ROWS THEN
    RAISE ...
@matthew-n
matthew-n / pg_unpivot_ex.sql
Created April 22, 2017 18:46
postgresql unpivot with hstore example
WITH fedex_rates_ip
("fid","Type","Weight","ZoneA","ZoneB","ZoneC","ZoneD","ZoneE","ZoneF","ZoneG",
"ZoneH","ZoneI","ZoneJ","ZoneK","ZoneL","ZoneM","ZoneN","ZoneO","ZonePuertoRico")
as(
select
*
from (
values
(2,'IntlPriority',0,40.25,41.5,43,54.75,116.5,52,51.5,61.25,49.5,63.5,62.5,82,119.25,61,63.25,36.65),
(3,'IntlPriority',-1,66.25,67.75,62.25,74.25,132,68,68.25,85.75,66.25,84.5,82.25,99.5,136.5,79.75,85.5,null),
@matthew-n
matthew-n / postgresql_grid.sql
Last active February 2, 2017 14:28 — forked from thiagomata/postgresql_grid.sql
reduce vars
/*
-- EXAMPLE --
SELECT
*
FROM grid(array(SELECT
(
lat,
@matthew-n
matthew-n / bit_flag_fun.sql
Created June 29, 2016 19:14
dealing with legacy bit encoding table
-- sparce matrix of (baz, biz, buz)
CREATE TABLE myLegacyBitFlagTbl(
baz char(2) NOT NULL,
biz char(2) NOT NULL,
-- bit flag fields encode buz(int) values
colA INT NOT NULL, --LSB in big-endian
colB INT NOT NULL,