jimfulton’s gists

jimfulton / bqupload.py

Created July 16, 2021 20:11

(grungy :)) Script to time large test BigQuery load

	import sys
	import time
	import contextlib
	from pprint import pprint as pp
	from google.cloud import bigquery
	client = bigquery.Client()

	def cq(sql, a, *k):
	pp(list(client.query(sql, a, *k)))

jimfulton / Main.java

Last active June 30, 2021 19:16

Update BigTable with reverse timestamps, https://github.com/googleapis/google-cloud-go/issues/4087

	package dev.j1m;

	import com.google.cloud.bigtable.hbase.BigtableConfiguration;
	import org.apache.hadoop.hbase.Cell;
	import org.apache.hadoop.hbase.TableName;
	import org.apache.hadoop.hbase.client.*;
	import org.apache.hadoop.hbase.util.Bytes;
	import java.io.IOException;
	import java.nio.ByteBuffer;

jimfulton / test-hierarchy-search-in-trigger.py

Created June 26, 2017 22:50

Test script that used triggers to populate hierarchical

	from pprint import pprint
	import psycopg2
	import uuid
	from contextlib import closing

	setup_sql = """
	create table data (id int primary key, state jsonb, isc bool);
	create table stage (id int primary key, state jsonb, isc bool);

	create or replace function find_cid(fid int, fstate jsonb, fisc bool)

jimfulton / buildout.cfg

Created April 19, 2017 12:53

ZEO server buildout configuration example

	[buildout]
	parts = server

	[server]
	=> zeo config
	recipe = zc.zdaemonrecipe
	program = ${buildout:bin-directory}/runzeo -C ${config:location}

	[zeo]
	recipe = zc.recipe.egg

jimfulton / zc.cfg

Created April 1, 2017 20:03

An attempt at a configuration file for the mysterious zeoctl

jimfulton / A GIN DDL and Schema

Last active August 16, 2017 17:47

Comparison of GIN and RUM indexes for ranked searches of a database with ~220K documents and searches returning ~50K documents

	=> create index pgtextindex_text_vector_idx on pgtextindex using gin (text_vector);
	CREATE INDEX
	=> analyze pgtextindex;
	ANALYZE
	=> \d pgtextindex
	Table "public.pgtextindex"
	Column \| Type \| Modifiers
	-------------------+-----------------------------+----------------------
	docid \| integer \| not null
	community_docid \| character varying(100) \|

jimfulton / Environment.txt

Created January 11, 2017 22:17

ZODB shootout results with and without a prepared statement for load_current

Non-history-preserving storage Python 3 on Mac OS X 10.10.4 2.3 GHz Intel Core i7 16GB, Postgres 9.5

jimfulton / fs1b.rs

Created September 25, 2016 14:06

Rust script for iterating over a ZODB file storage file using a single iterator yielding an enum type

	/// Summarize a file-storage 1 formatted file, mainly as a way of
	/// gaging rust performance. This is more or less eqivalent to the
	/// following Python/ZODB script:
	///
	/// import ZODB.FileStorage
	/// import sys
	///
	/// def main():
	/// it = ZODB.FileStorage.FileIterator(sys.argv[1])
	/// transactions = records = 0

jimfulton / fs1.rs

Last active September 25, 2016 14:05

Rust script for iterating over a ZODB file storage file using nested iterators.

	/// Summarize a file-storage 1 formatted file, mainly as a way of
	/// gaging rust performance. This is more or less eqivalent to the
	/// following Python/ZODB script:
	///
	/// import ZODB.FileStorage
	/// import sys
	///
	/// def main():
	/// it = ZODB.FileStorage.FileIterator(sys.argv[1])
	/// transactions = records = 0

jimfulton / perf4.py

Created August 10, 2016 18:58

A zodbshootout inspired by ZEO-specific performance-measurement script.

	import argparse
	import logging
	import multiprocessing
	import os
	import persistent
	import re
	import time

	import ZODB
	from BTrees.IOBTree import BTree

Jim Fulton jimfulton