Daniel Selans dselans

How-to: Migrating repos on Github to a monorepo

To avoid peppering too many opinions in this “how to” guide, I’ve written a separate “opinion-piece” article on monorepos — you can read it here: https://medium.com/streamdal/mostly-terrible-the-monorepo-5db704f76bdb

Sometime in early Dec 2023, our team decided to migrate ~10 public repositories for an OSS project to a monorepo.

This article provides a “rough” outline for the meatiest parts of the migration.

Good luck! 🤞

I compiled these while learning about running Apache Pulsar. The notes are written from the perspective of a Go developer that interfaces primarily with Kafka, RabbitMQ and NATS. Something may not be entirely correct. Sorry!

Concept of “bundles”
Each broker is in charge of certain bundles
A bundle represents multiple topics
Pulsar has a tunable automatic load shedder
Possible to configure to auto shed bundles to less loaded brokers
Bundles are automatically split when under heavy load
Bundle is split into more bundles

I wasn't able to find any concrete examples of folks running cassandra-stress against TLS + auth enabled cassandra clusters; this gist hopefully fulfills that.

Setup

3 node C* setup w/ lucene running on instaclustr
TLS Enabled
Auth Enabled

	package main

	import (
	"context"
	"fmt"
	"log"
	"math/rand"
	"time"

	streamdal "github.com/streamdal/go-sdk"

	# Took a good hour+ to get this going as there's a lack of docs on how to do this (and/or I don't know where to look)
	#
	# Download the better mc server pack zip file from curseforge. Move that to /root/minecraft/ggf-fabric-bmc/pack.
	# Reference the zip file in GENERIC_PACK.
	# docker-compose up -d and watch the logs. There will be lots of errors due to better mc using ~200 mods but it should start
	# nevertheless.

	bt-minecraft:
	image: itzg/minecraft-server
	container_name: ggf-fabric-bmc-fixed

	// Quick example of using the headers exchange with the streadway/amqp lib.
	//
	// NOTE: I couldn't find any examples of this anywhere, so maybe this will help
	// someone else.
	//
	// Run consumer in one terminal: $ go run main.go
	// Run publisher in another terminal: $ go run main.go -action publisher
	//
	// The consumer should have received a message.
	//

	// Launches 100 goroutines to mass delete keys that match certain criteria.
	//
	// Current results: ~3,600 keys/s (~193,000 keys/minute)
	package main

	import (
	"fmt"
	"os"
	"time"

	#!/usr/bin/env python
	#

	import re
	import sys
	import pickle
	import pprint
	import feedparser

	from slackclient import SlackClient

	// example reverse proxy
	package main

	import (
	"fmt"
	"log"
	"net/http"
	"net/http/httputil"
	)

Daniel Selans dselans

How-to: Migrating repos on Github to a monorepo

Setup

Run