Skip to content

Instantly share code, notes, and snippets.

View lawrencejones's full-sized avatar

Lawrence Jones lawrencejones

View GitHub Profile
@lawrencejones
lawrencejones / looker.bash
Last active March 26, 2020 16:16
Looker API helper
# Source this file ($ source <file-namne>) in your terminal to load all the
# functions.
# First run the looker-login using the client ID and client secret to generate
# an API token. Once you have your API token, run:
#
# $ export LOOKER_API_TOKEN=<your-token>
#
# This will enable use of looker-curl.
@lawrencejones
lawrencejones / verify.rb
Created December 12, 2018 13:50
Fiddling with secret key svc2svc auth
#!/usr/bin/env ruby
require "securerandom"
require "benchmark"
require "openssl"
class Svc2Svc
def initialize(count: 1000)
@keys = Array.new(1000) { SecureRandom.hex }
@services = keys.each_with_object({paysvc_live: [], paysvc_sandbox: [], banking: []}) do |key, services|
@lawrencejones
lawrencejones / run.py
Created November 23, 2018 12:20
BigQuery to Postgres for feature repo
import sqlalchemy
from sqlalchemy import Column, Integer, String, Table, MetaData, DateTime
from sqlalchemy.sql import func
from sqlalchemy.dialects.postgresql import insert, JSONB
from google.cloud import bigquery
from itertools import islice, chain
def batch(iterable, batch_size=500):
source = iter(iterable)
while True:
package main
import (
"bytes"
"compress/gzip"
"encoding/base64"
"fmt"
"io/ioutil"
"os"
@lawrencejones
lawrencejones / serialize.rb
Created July 26, 2018 13:55
compute_serialized_associations
# Given an active record element and a callable that serializes it, this method will
# compute the associations that are required by the serializer. We produce a single
# hash that looks almost like what #eager_load will accept, except terminating
# relations are represented by key: {} (empty hash).
#
# elem = Customer.last
# serialize = -> { API::Search::CustomerSerializer.new(elem).as_json }
#
# compute_loaded_associations(elem, serialize) => {
# bank_accounts: { mandates: { subscriptions: {} } },
@lawrencejones
lawrencejones / keys.rb
Created May 1, 2018 14:55
Use jaccard index to identify log lines that are similar to one another
#!/usr/bin/env ruby
require "json"
=begin
[2018-04-30T00:00:57,316][WARN ][logstash.outputs.elasticsearch] Failed action. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash-2018.04.30", :_type=>"logs", :_routing=>nil}, 2018-04-30T00:00:57.000Z app01 usage_tracker], :response=>{"index"=>{"_index"=>"logstash-2018.04.30", "_type"=>"logs", "_id"=>"AWMT2jPQZUVqEe-MfAiF", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [id]", "caused_by"=>{"type"=>"number_format_exception", "reason"=>"For input string: \"PM00096PW1GVDT\""}}}}})
=end
threshold = (ARGV.first || "0.7").to_f
puts("Clustering with >#{threshold} Jaccard index...\n")

Something

Something dangerous

- This is my warning
- my warning is important
- heed this warning
#!/usr/bin/env ruby
require "tempfile"
require "open3"
require "shellwords"
require "yaml"
require "base64"
require "active_support/core_ext/hash"
def print_usage
@lawrencejones
lawrencejones / App.Makefile
Created November 30, 2017 14:07
Example makefile template, slightly mangled file structure
# == Building Apps ==
#
# Provides several targets, which do the following...
#
# all: builds all target applications
# clean: removes any binaries, .o or final
#
# == Creating New Apps ==
#
# To define a Makefile for a new application, the apps Makefile need only
@lawrencejones
lawrencejones / matrixMultiply.ast
Created November 24, 2017 16:53
Matrix multiplication AST from Wacc source code
Program
[]
[ Declare
(ArrayType (ArrayType (Type "int")))
(Ident "A")
(ArrayLit
[ ArrayLit [ Unary (Op "-") (IntLit 2) , IntLit 5 ]
, ArrayLit [ IntLit 3 , IntLit 7 ]
])
, Declare