Skip to content

Instantly share code, notes, and snippets.

View darkblue-b's full-sized avatar
👁️
.

Brian M Hamlin darkblue-b

👁️
.
View GitHub Profile
@darkblue-b
darkblue-b / parse_log24.py
Last active December 20, 2015 01:29
next rev to parse build log
import sys,re
# more hacks
# remove the last two lines from the saved output
# psql
# > create database pkgs1;
# > \c pkgs1
# pkgs1=# create table raw_parse (script_name text, action text, pkg_name text);
# pkgs1=# copy raw_parse from '/home/user/out1.csv' with CSV delimiter E'\t';
# pkgs1=# select pkg_name from raw_parse group by pkg_name having every(action <> 'REMOVE') order by pkg_name;
@darkblue-b
darkblue-b / import_master_20121yr_8.py
Last active December 24, 2015 22:59
imports ACS 2012 1yr extension - rev8: first cut at geodata; imports multiple states as previously; needs more QA
#!/usr/bin/python
##===========================================================
## Import ACS 2012 1 year extension
## GPLv3 Oct 2013 -dbb
##
## Useage: python import_master_20121yr_8.py Sequence_Number_and_Table_Number_Lookup.txt work_dir
## assumes an existing database 'census_acs_2012' ; currently writes to public schema
##
## Note: results in ~1480 tables plus 1478 more per state, so
## max_locks_per_transaction must be manually increased
@darkblue-b
darkblue-b / import_master_20121yr_9.py
Last active December 25, 2015 00:39
imports ACS 2012 1yr extension - rev9: has geodata fkeys; imports multiple states as previously; added 'c' continue load option
#!/usr/bin/python
##===========================================================
## Import ACS 2012 1 year extension
## GPLv3 Oct 2013 -dbb
##
## Useage: python import_master_20121yr_9.py Sequence_Number_and_Table_Number_Lookup.txt work_dir
## assumes an existing database 'census_acs_2012' ; currently writes to public schema
##
## Note: results in ~1480 tables plus 1478 more per state, so
## max_locks_per_transaction must be manually increased
@darkblue-b
darkblue-b / do_make_region_grids.py
Last active December 25, 2015 13:49
classify a list of geometries via table; extract grid cells for each class; build indexes and add pkeys
tSQL = '''
drop table if exists region_{0}_grid cascade;
create table region_{0}_grid as
select distinct on (g.gridcode) g.gridcode as gid,
st_transform( ST_GeometryN(g.the_geom,1), 4326)::geometry(Polygon,4326) as geom
from grid150m_uniq g,
tl_2010_06_county10_3310 c, region_defs r
where st_intersects( g.the_geom, c.geom ) AND
c.countyfp10 = r.fips AND
r.region_id = {0};
@darkblue-b
darkblue-b / t2.py
Created October 19, 2013 02:36
sample data with a certain distribution
import psycopg2
import random
conn = psycopg2.connect( 'dbname=ta_test' )
curs = conn.cursor()
tSQL_create0 = '''
drop table if exists tta cascade;
create table tta ( tta_item integer, pkey serial);
#!/usr/bin/python
# -*- coding: utf-8 -*-
# Copyright (c) 2013 Brian Hamlin - darkblueb
# Copyright (c) 2013 The Open Source Geospatial Foundation.
# Licensed under the GNU LGPL version >= 2.1.
#
# This library is free software; you can redistribute it and/or modify it
# under the terms of the GNU Lesser General Public License as published
# by the Free Software Foundation, either version 2.1 of the License,
import psycopg2
import time
conn = psycopg2.connect("dbname=text_test0")
curs = conn.cursor()
base_chars = ( 'A','C','G','T','U')
base_strs_t = (
'AACGTAACGTAACGTAACGTAACGTAACGTAACGTAACGT',
@darkblue-b
darkblue-b / SanMateo_CA_IRIS_0.ipynb
Created January 16, 2015 08:12
IRIS plot of elevation data in netCDF form
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@darkblue-b
darkblue-b / live85_py.txt
Created January 17, 2015 04:38
Live 8.5 python by rough category
cartopy,click
imposm
iris
descartes
eoxserver
fiona
geoexplorer,geolinks
geopandas,geoserver
gps,geonode
kmlbase,kmldom,kmlengine
@darkblue-b
darkblue-b / cartopy-features-test0.ipynb
Created January 22, 2015 08:01
a notebook test of cartopy cached features
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.