Skip to content

Instantly share code, notes, and snippets.

View pdlug's full-sized avatar

Paul Dlug pdlug

View GitHub Profile

Tested two potential optimizations based on a comment on the TypeGraph post in /r/postgresql from u/Deep_Ad1959:

  1. Native CYCLE clause vs the manual ARRAY[] + != ALL(path) cycle detection that TypeGraph currently emits (packages/typegraph/src/query/dialect/postgres.ts:178-192).
  2. Prepared-statement plan caching can pick a bad generic plan after the 5th execution under plan_cache_mode = auto when start-node selectivity varies widely.

Setup

Draft of a post on TypeGraph performance. Eager to get feedback on it, reach me at: paul@nicia.ai, X: @pdlug

How fast is TypeGraph? (draft)

If TypeGraph compiles to SQL, doesn't a real graph database win on performance? Sometimes. Usually not. For most workloads — thousands to millions of nodes, traversals up to 5–10 hops deep, the queries that show up in real applications — the comparison is closer than graph-DB marketing suggests, because every TypeGraph query compiles to a single SQL statement that the database plans, executes, and streams back as one result set. There is no dataloader, no batching layer, nothing to amortize.

A three-way head-to-head: TypeGraph on SQLite (in-process via better-sqlite3), TypeGraph on PostgreSQL 16, and Neo4j 5.26. Same machine, identical methodology, full harness in packages/benchmarks. Graph shape: 1,200 users, 10 follow

@pdlug
pdlug / index.ts
Last active October 21, 2018 19:05
Example of embedding Apollo GraphQL server inside next.js
import * as express from 'express'
import * as next from 'next'
import { ApolloServer, gql } from 'apollo-server-express'
const port = parseInt(process.env.PORT, 10) || 3000
const dev = process.env.NODE_ENV !== 'production';
const app = next({ dev, quiet: false })
const handle = app.getRequestHandler()
@pdlug
pdlug / contribs.clj
Created February 4, 2015 04:59
Demo of parsing JATS XML using `clojure.data.zip.xml` to extract contributors.
(ns jats.contribs
(:require [clojure.java.io :as io]
[clojure.zip :as zip]
[clojure.data.xml :as xml]
[clojure.data.zip.xml :as zx]
[clojure.pprint :as pprint])
(:gen-class :main true))
(defn contrib-name [name]
(let [first-name (zx/xml1-> name :given-names zx/text)
# Paul's ZSH Theme, based on several others
PROMPT='$(_user_host)${_current_dir}$(git_prompt_info)%{$reset_color%}${_return_status}%# '
local _current_dir="%3~ "
local _return_status="%{$fg[red]%}%(?..⍉)%{$reset_color%}"
local _hist_no="%{$fg[grey]%}%h%{$reset_color%}"
function _user_host() {
if [[ -n $SSH_CONNECTION ]]; then
@pdlug
pdlug / pacer_loop_test.rb
Created July 20, 2011 16:20
Pacer loop test
require 'bundler/setup'
require 'pacer'
require 'pacer-neo4j'
#graph = Pacer.tg
graph = Pacer.neo4j('/tmp/test-neo4j')
c1 = graph.create_vertex(name: 'Concept 1')
c2 = graph.create_vertex(name: 'Concept 2')
@pdlug
pdlug / gist:1072288
Created July 8, 2011 17:11
Test AMQP consumer/producer with auto recovery
# CONSUMER:
require 'amqp'
def setup_connection(connection)
puts "Setting up connection"
channel = AMQP::Channel.new(connection, :auto_recovery => true)
channel.on_error do |ch, channel_close|
raise channel_close.reply_text
end
@pdlug
pdlug / gist:962855
Created May 9, 2011 16:45
AMQP connection failure handling
def setup_connection(connection)
# open channel, set up exchanges and bindings, subscribe
end
AMQP.start(
:host => '127.0.0.1',
:vhost => '/search',
:timeout => 0.3,
:on_tcp_connection_failure => Proc.new { |settings| puts "Failed to connect"; EM.stop }) do |connection, open_ok|
describe "when handling results" do
before(:each) do
AMQP.stub!(:connect)
MQ.stub!(:new)
Nanite::MapperProxy.new('mapperproxy', {})
@instance = Nanite::MapperProxy.instance
@fanout = stub(:fanout, :publish => true)
@instance.amqp.stub!(:fanout).and_return(@fanout)
@payload = {:payload => ['nanite', 'eventmachine', 'rabbitmq']}
end
require 'rubygems'
gem 'dm-core'
gem 'dm-types'
require 'dm-core'
require 'dm-types'
class User
include DataMapper::Resource