Skip to content

Instantly share code, notes, and snippets.

View luketn's full-sized avatar

Luke Thompson luketn

31 followers · 40 following
View GitHub Profile
@luketn
luketn / lucene-luke.md
Last active April 28, 2025 22:28 — forked from vladak/lucene-luke.md
How to run Luke

Lucene has Luke built in - to run it from the code I use this command:

git clone https://github.com/apache/lucene.git Lucene
cd Lucene
./gradlew assemble
java -cp './lucene/luke/build/lucene-luke-11.0.0-SNAPSHOT/*' \
    -Xmx1024m -Xms512m -XX:MaxMetaspaceSize=256m org.apache.lucene.luke.app.desktop.LukeMain
@luketn
luketn / hnsw-diagram.html
Created March 31, 2024 08:02
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>HNSW Graph Visualization</title>
<script src="https://d3js.org/d3.v7.min.js"></script>
<style>
.link {
stroke: #999;
stroke-opacity: 0.6;
@luketn
luketn / gist:99302cf20039a56772ff9eef707dede7
Created March 15, 2024 13:20
Install Docker and Docker Compose on AL2023
dnf install -y docker git
DOCKER_CONFIG=${DOCKER_CONFIG:-$HOME/.docker}
mkdir -p $DOCKER_CONFIG/cli-plugins
curl -SL https://github.com/docker/compose/releases/download/v2.24.7/docker-compose-linux-x86_64 -o $DOCKER_CONFIG/cli-plugins/docker-compose
chmod +x $DOCKER_CONFIG/cli-plugins/docker-compose
systemctl start docker
@luketn
luketn / cdntest.js
Created November 18, 2022 04:18
CDN Test Script
var API_PREFIX = '/api/v1';
var stats = [];
var TEST_CDN = '';
function recordStat(url, cdnName, statName, timeInMillis) {
stats.push({
"name": statName,
"cdn": cdnName,
"timeTakenMillis": timeInMillis
});
@luketn
luketn / mongo-trim.sh
Created October 28, 2022 07:49
Create Trimmed Mongo node_modules (excluding the AWS SDK) for deployment to Lambda
#!/usr/bin/env bash
set -e
rm -rf node_modules
npm init -y
npm install mongodb --production
echo "before: $(du -sh node_modules)"
rm -rf node_modules/@aws-sdk
rm -rf node_modules/@aws-crypto
@luketn
luketn / run-mongo-replica-set.sh
Last active September 27, 2022 05:10
Start MongoDB docker image as a replica set
#!/bin/sh
# Map input values from the GitHub Actions workflow to shell variables
MONGODB_VERSION="6.0"
MONGODB_REPLICA_SET="test-rs"
MONGODB_PORT="27017"
MONGO_CMD="mongosh" # for earlier versions use "mongo"
if [ -z "$MONGODB_VERSION" ]; then
echo ""
@luketn
luketn / Dockerfile
Created September 14, 2022 07:14
Serverless MongoDB Docker Lambda Import/Export Sync
# Ref: https://gallery.ecr.aws/lambda/provided
# + https://docs.aws.amazon.com/lambda/latest/dg/runtimes-images.html
# + https://aws.amazon.com/blogs/aws/new-for-aws-lambda-container-image-support/
FROM public.ecr.aws/lambda/provided:al2
COPY bootstrap.sh ${LAMBDA_RUNTIME_DIR}/bootstrap
COPY function.sh ${LAMBDA_TASK_ROOT}
# Ref https://www.mongodb.com/docs/v6.0/tutorial/install-mongodb-on-amazon/
SHELL ["/bin/bash", "-c"]
@luketn
luketn / Boto3 Typed in PyCharm.md
Created February 3, 2022 04:13
This is how you set up PyCharm for Boto3 python development with types.
@luketn
luketn / spark.java
Created November 20, 2021 06:08
Convert CSV to Parquet
package com.thing;
import com.google.common.io.Files;
import org.apache.spark.sql.Dataset;
import org.apache.spark.sql.Row;
import org.apache.spark.sql.SparkSession;
import lombok.extern.slf4j.Slf4j;
import java.awt.*;
import java.io.File;
@luketn
luketn / Install Docker and Docker-Compose on AWS Linux 2, then Run a Load Test
Last active September 6, 2021 22:55
# Switch to privileged user 'root'
sudo su
# Install Docker + git, enabling the docker process to start on boot
yum install -y docker git
systemctl enable --now docker
# Install Docker-Compose
curl -L https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m) -o /usr/bin/docker-compose
chmod +x /usr/bin/docker-compose