Skip to content

Instantly share code, notes, and snippets.

@jrudolph
jrudolph / orientation.md
Last active September 21, 2026 06:19
Vesuvius Challenge Open Data Volume Orientation Bits

Volume Orientation Properties

Three optional fields in a volume's properties record how its voxel frame relates to the physical object. They are defined per volume but can be propagated / cross-checked using affine transforms between volumes. When the orientation is correct, segments are oriented in proper reading order without mirroring.

Property Values Meaning
left_handed_coordinates true/false Volume coordinate system handedness.
@jrudolph
jrudolph / cgroup-io-limits.md
Created March 11, 2025 15:11
Limiting IO bandwidth of processes with cgroup in Linux

Here's a way to limit the IO usage of a process (or process group).

What did not really make a difference is using ionice to change the priority. In this case, the reason might be, that the background job is a highly concurrent rclone job and it might still dominate the IO queues by the sheer amount of concurrent IO scheduled.

What does work, is using cgroups to limit the total IO bandwidth for a group.

Create a group:

@jrudolph
jrudolph / fiber-graph.md
Last active February 28, 2025 13:57
Notes on creating a mesh from fiber crosspoints

Creating segments from intersecting horizontal and vertical fibers

Sean published results of his volume segmentation models that can detect and distinguish horizontal and vertical fibers (first in Ostober 2024 and later in January 2025)

Idea: recreate the fiber mesh by finding and connecting crosspoint between horizontal and vertical fibers

Process

  • Separate the fibers by running a connected component analysis over the model output, separately for horizontal and vertical fibers
@jrudolph
jrudolph / delete-local-branches.sh
Created May 17, 2024 07:33
Delete local branches that have already been squash merged into origin/main
# adapted from https://github.com/not-an-aardvark/git-delete-squashed
git checkout -q origin/main && git for-each-ref refs/heads/ "--format=%(refname:short)" | \
while read branch; do mergeBase=$(git merge-base origin/main $branch) && \
[[ $(git cherry origin/main $(git commit-tree $(git rev-parse "$branch^{tree}") -p $mergeBase -m _)) == "-"* ]] && \
git branch -D $branch; \
done
@jrudolph
jrudolph / ppm.md
Last active October 31, 2023 13:01
Volume Cartographer PPM file format

Volume Cartographer PPM file format

PPM files map U/V coordinates from flattened surfaces back to the original 3D volume coordinates x/y/z and also provide a normal for every point.

The file has a small header and is otherwise a huge array of double values (in the common case, but see header).

Header

@jrudolph
jrudolph / FreiburgImBreisgau.md
Created August 15, 2023 12:55
Freiburg im Breisgau (as imagined by llama-2-7b.ggmlv3.q4_0.bin)

Freiburg im Breisgau

The town of Freiburg is located in south-western Germany, in the state of Baden-Württemberg, 150 km (93 mi) south-west of Karlsruhe, 110 km (68 mi) west of Strasbourg, France, and 130 km (81 mi) east of the Alpine passes of the Alps.

Geography

Freiburg lies in the extreme south of Baden-Württemberg and is bordered on the west by France. It lies 500 m (1,600 ft) above sea level, within the southern part of the northern Black Forest at the foothills of the Alps, near the confluence of the rivers Singold and Dreisam, in the historical region of Sundgau, locally referred to as Breisgau. Freiburg is in the traditional wine-growing region of Germany and hosts one of the country's largest wine festivals, the Freiburger Weihnachtsmarkt, which takes place from the last weekend in November until the first weekend in December.

Subdivisions

@jrudolph
jrudolph / suggestions.txt
Created August 12, 2023 21:27
Evaluate newhope model on llama2.scala to find algorithmic improvements to its nucleus sampling method
Model from https://huggingface.co/TheBloke/NewHope-GGML
llama2JVM ### Instruction:
llama2JVM Generate scala code that find the the top-p (for nucleus sampling) elements, i.e. the smallest set of elements that reaches a given level `p` of cumulative probability, of an unsorted array of probabilities summing to 1. Expect a power-law distribution, i.e. the number of elements found will be low (< 10). Do not use any data structures besides arrays (in particular, expect that inserting into apriority queue will be too slow). Avoid sorting.
llama2JVM
llama2JVM ### Response:
llama2JVM Here's a possible implementation in Scala that uses binary search and a priority queue to solve this problem.
llama2JVM
llama2JVM ```scala
llama2JVM import scala.annotation.tailrec
@jrudolph
jrudolph / FoldAllocationsForFlameGraph.scala
Created May 3, 2022 09:46
Generate allocation flamegraph from JFR
import akka.actor.ActorSystem
import akka.stream.{ Attributes, FlowShape, Inlet, Outlet, OverflowStrategy }
import akka.stream.scaladsl.{ Compression, FileIO, JsonFraming }
import akka.stream.stage.{ GraphStage, GraphStageLogic, InHandler, OutHandler }
import akka.util.ByteString
import java.io.{ File, FileOutputStream }
import spray.json._
import scala.concurrent.Future
@jrudolph
jrudolph / yugabyte-debugging.md
Last active September 12, 2022 09:27
Yugabyte debugging

Yugabyte Debugging Case Study

Problem

Under load, the yugabyte clusters showed a latency spike ~ every 30 seconds. Latencies went up to 4 seconds.

Analysis

Aside from the main problem, we found that one of the servers in the cluster showed CPU saturation during those spikes. Since the spikes were frequent enough, top could be used to identify the yb-master process as the

@jrudolph
jrudolph / yb-architecture.md
Created April 7, 2022 09:32
Yugabyte Architecture / Implementation Details

As of version 2.12.1.0

https://docs.yugabyte.com/preview/architecture/ has some information but too little about the actual implementation details.

The whole project is written in C++.

Components

  • master(s): provide metadata services and are involved in all kinds of metadata changes, manage the set of tservers, disitrbute tablets to tservers. There's a cluster of masters and RAFT is used to determine a leader. It seems that,