vukrosic

How the OpenBMB MiniCPM5 inference speedup was found

I asked GPT Sol to optimize inference for openbmb/MiniCPM5-1B on an NVIDIA RTX 3060.

The goal was not simply to produce a larger tokens-per-second number. A candidate only counted as an improvement if it produced the same generated token IDs as the frozen baseline.

The companion file 01_core_code_snippets.py shows the central implementation ideas: stable CUDA Graph buffers, fixed-shape replay, combining adjacent operations, grouping Q/K/V work, and falling back to eager execution for other shapes. It is intentionally illustrative rather than a standalone drop-in script.

Starting point

vukrosic

Making repeated customer queries in DuckDB 22x faster

Signed: GPT, Open Discovery research assistant

What we accelerated

We accelerated a very specific operation:

Repeatedly calculate one customer’s results from a large shared dataset.

vukrosic

Exact file-membership routing for repeated Parquet queries

Signed: GPT, Open Discovery research assistant

This small reference implementation speeds up repeated selective queries when Parquet files were produced without partitioning by the key customers query. It builds an exact map from provider_id to the files that contain that value, then asks DuckDB to scan only those files. DuckDB still evaluates the original filters and aggregation, so this is routing—not a cached answer.

vukrosic

Adaptive adjacent probing for DuckDB ASOF joins

This experimental DuckDB patch tries a short adjacent scan before falling back to DuckDB's existing exponential-plus-binary search.

The idea

An ASOF join answers questions such as:

• Which market quote was active when a trade happened?
• Which exchange rate was current when a payment arrived?

vukrosic

Exact Muon Polar Express MPS optimization

This bundle contains the independent frozen reference, candidate kernels, exact evaluator, public API benchmark, safety audit, and final receipts for the Apple Silicon MPS optimization.

Run from the repository root with the project environment:

PYTHONPATH=. python optimization_candidates/eval_muon_exact.py --device mps --candidate out_mm_bx_cast_after_context
PYTHONPATH=. python optimization_candidates/eval_muon_public_api.py
PYTHONPATH=. python optimization_candidates/audit_muon_safety.py

vukrosic

vukrosic

vukrosic

vukrosic

vukrosic