DBI-SuperCollider

README.md

DBI SuperCollider MVP Plan

Goal

Demonstrate SuperCollider-style instrumentation of AMDGPU LDS accesses under rocJITsu DBI.

The concrete deliverable is an opt-in rocJITsu instrumentation mode that can catch at least some LDS races on selected tests. This is intentionally a race-exposure MVP, not a full happens-before sanitizer.

Guiding Constraints

• Prefer the existing HSA tools path if it is practical.
• Fall back to a waitcheck-style LD_PRELOAD path only if HSA-level interception blocks first-light progress.
• Treat hip-moi as the innermost iteration corpus:
  • source: ~/workspace/hip-moi
  • build: ~/workspace/hip-moi-build
• Treat IREE as periodic broader coverage, not the tight loop:
  • source: ~/workspace/iree
  • build: ~/workspace/iree-build
  • run a focused IREE subset roughly once per 30-minute session.
• The local GPU target is RDNA4 / gfx1201. This MVP is instrumentation-only: rocJITsu should patch the loaded RDNA4 code object in place, not translate it to another architecture.
• When running GPU tests, cap parallelism at roughly 8 jobs.
• Split work into chunks that fit roughly into 30-minute sessions.
• Achieve very minimal first-light within about 6 hours.
• Deliver the useful MVP later this week, roughly within 2 days of focused work.

MVP Semantics

Instrument ordinary non-atomic LDS reads and writes with the SuperCollider redundant-access idea:

• For an LDS read:
  • perform the original read,
  • delay for a small randomized or configurable window,
  • read the same LDS address again,
  • report if the two values differ.
• For an LDS write:
  • perform the original write,
  • delay for a small randomized or configurable window,
  • read the same LDS address back,
  • report if the observed value differs from the value written.

The first implementation may report by trapping. A report counter is preferred once basic binary patching is proven.

Initial Scope

Support first:

• LDS/shared-memory ds_read* and ds_write*.
• Common fixed-width forms, starting with b32, then b64, then b128.
• Known test kernels from a tiny synthetic HIP test, hip-moi, and IREE.
• Opt-in instrumentation controlled by environment variables.

Reject, skip, or route away at first:

• ds_*atomic.
• global_atomic, flat_atomic, buffer_atomic, and s_atomic.
• Explicit cache/fence-like opcodes such as global_wb, global_inv, buffer_wbinvl1, and s_dcache.
• Global memory instrumentation.
• Async copies.
• Full barrier epoch modeling.
• Full per-address shadow memory.
• Precise source-level reporting.

Unsupported cases must produce clear diagnostics. A clean run must not imply coverage when the preflight rejected the kernel.

Preferred Architecture

Use the HSA tools hook first.

Expected invocation shape:

HSA_TOOLS_LIB=/path/to/librocjitsu_dbi_hooks.so \
RJ_DBI_SUPERCOLLIDER=1 \
RJ_DBI_LOG=1 \
RJ_DBI_SC_DELAY=32 \
./app

Proposed DBI environment variables:

• RJ_DBI_SUPERCOLLIDER=1: enable this instrumentation mode.
• RJ_DBI_LOG=1: print code-object, kernel, patch, skip, and report diagnostics.
• RJ_DBI_SC_DELAY=N: request an approximate delay between the original LDS access and the duplicate/readback access. This is not an existing rocJITsu variable yet. For the first MVP, define it as the number of emitted simple NOP delay instructions. It should not be documented as exact GPU cycles. Later versions can add a scalar loop or randomized delay window if plain NOPs are not enough to expose races.
• RJ_DBI_FAIL_CLOSED=1: optional debug mode that rejects kernels with unsupported LDS/fence/atomic features instead of silently skipping them.

Implementation shape:

1. Extend the existing rocJITsu HSA tools hook with an opt-in DBI mode.
2. Reuse the memory-backed code-object reader registry already used by the DBT hook.
3. At hsa_executable_load_agent_code_object, pass recorded ELF bytes through a DBI SuperCollider patching path.
4. Create a replacement memory-backed reader over patched bytes.
5. Call the saved original HSA load function with the replacement reader.

Fallback shape:

• If memory-backed reader coverage is insufficient, add a waitcheck-style LD_PRELOAD shim that intercepts code-object reader creation earlier, including file and AMD loader extension-table paths.

Source Placement Reconnaissance

This is the current best map of where the DBI SuperCollider MVP should live in the rocJITsu tree.

New Source Files

Core patching should start in the existing DBI patch layer:

• lib/rocjitsu/src/rocjitsu/code/patch/supercollider_dbi.h
• lib/rocjitsu/src/rocjitsu/code/patch/supercollider_dbi.cpp

This should own the opt-in patch entry point and the MVP data model:

• SuperColliderDbiOptions
• SuperColliderDbiResult
• LdsAccessKind
• LdsAccessSite
• scan/classify helpers for ds_read*, ds_write*, ds_*atomic, barriers, and fence-like instructions
• the first patching entry point that takes an AmdGpuCodeObject plus options and returns patched bytes or diagnostics

Keep this separate from patch/instrumentor.{h,cpp} initially. The existing Instrumentor is a good relocation/trampoline smoke-test path, but it is still deliberately constrained around BeforeInst and inline s_nop insertion. It should remain the minimal DBI probe path until the SuperCollider code proves the richer replacement sequences.

If supercollider_dbi.cpp grows too quickly, split it later into:

• lib/rocjitsu/src/rocjitsu/code/patch/lds_access_classifier.{h,cpp}
• lib/rocjitsu/src/rocjitsu/code/patch/supercollider_emitter.{h,cpp}

The HSA interception source should preferably be a distinct tools library:

• lib/rocjitsu/src/rocjitsu/hooks/rj_hsa_dbi_hooks.cpp

This file can reuse the same HSA tools mechanism as rj_hsa_dbt_hooks.cpp, but with DBI-oriented environment variables such as RJ_DBI_SUPERCOLLIDER=1. The fastest possible route is to extend rj_hsa_dbt_hooks.cpp in-place, but a separate rocjitsu_dbi_hooks target avoids tying DBI behavior to RJ_DBT_TARGET_ISA and avoids surprising existing DBT hook users. DBI should not ask rocJITsu to translate architectures for this MVP: on this machine, the loaded and patched target is always RDNA4 / gfx1201. If reader registry duplication becomes annoying, refactor the common HSA code-object reader registry into a small shared helper under hooks/.

Tests should be added alongside the current DBI and patch tests:

• tests/patch/supercollider_dbi_test.cpp
• tests/dbi/hsa_dbi_supercollider_test.cpp
• tests/kernels/lds_race_supercollider.hip

The first patch test should use synthetic or fixture code objects to prove classification and controlled patching. The HSA test should mirror the existing HSA-tools DBT smoke test shape, but set HSA_TOOLS_LIB to the new DBI hooks library.

Existing Files Likely Edited

Build glue:

• lib/rocjitsu/src/rocjitsu/code/CMakeLists.txt: add the new core SuperCollider patch source.
• lib/rocjitsu/src/rocjitsu/hooks/CMakeLists.txt: add a rocjitsu_dbi_hooks shared library target, or intentionally extend the existing rocjitsu_hooks target if we choose the fastest in-place path.
• tests/CMakeLists.txt: add unit tests and an HSA DBI hook CTest entry.
• tests/kernels/CMakeLists.txt: add the synthetic LDS race device kernel.

Instruction emission:

• lib/rocjitsu/src/rocjitsu/code/patch/instruction_builder.h
• lib/rocjitsu/src/rocjitsu/code/patch/instruction_builder.cpp

The first edit here is likely a small encoder for s_trap or another observable first-light instruction. Real SuperCollider checks will probably need additional vector/scalar compare, conditional branch, DS load/store, and reporting helpers unless an existing generated ISA encoder can be reused.

Descriptor/resource updates:

• lib/rocjitsu/src/rocjitsu/code/patch/code_object_patcher.{h,cpp}
• lib/rocjitsu/src/rocjitsu/code/dbt/kernel_descriptor_translator.{h,cpp}

These may not need edits for first-light. They become relevant once the MVP needs extra scratch, larger VGPR/SGPR allocation, or private segment growth. CodeObjectPatcher can already patch descriptor bytes, and DBT already has patterns for recomputing descriptor resource fields after liveness-selected scratch registers.

Register pressure helpers:

• Use analysis/liveness.{h,cpp} as-is if possible. It already provides find_free_run, find_free_sgpr, and find_free_sgpr_pair.
• Use patch/spill_manager.{h,cpp} as-is if possible. It already models a DBI spill zone appended to per-lane scratch.

These are likely dependencies of the SuperCollider path, not first edits.

Files To Avoid Editing Initially

• Avoid pushing SuperCollider-specific logic into patch/instrumentor.{h,cpp} until there is a clear shared abstraction with the existing inline-nop path.
• Avoid editing DBT semantic translator files for MVP DBI instrumentation. They are useful references for liveness and descriptor feedback, but the MVP should not couple itself to CDNA4-to-RDNA translation rules.
• Avoid touching IREE or hip-moi source during first-light; use them as test corpora once rocJITsu can patch a tiny local kernel.

First-Light Definition

First-light is complete when:

1. A test application runs under the rocJITsu HSA-level hook.
2. rocJITsu intercepts at least one code object.
3. rocJITsu identifies at least one LDS instruction.
4. rocJITsu patches the loaded binary in a visible way.
5. A deliberately racy tiny LDS test produces a report or trap.

First-light does not need to pass IREE or hip-moi yet.

MVP Definition Of Done

The MVP is complete when:

• rocJITsu has an opt-in DBI SuperCollider mode.
• The mode instruments some ordinary LDS ds_read* and ds_write* instructions.
• It catches at least one intentional LDS race.
• It runs against selected hip-moi and IREE tests.
• It emits clear patched/skipped/unsupported diagnostics.
• Unsupported atomics/fences are rejected or skipped explicitly.
• There is a short usage note documenting environment variables, known limitations, and test commands.

30-Minute Work Sessions

1. Inventory DBI And HSA Hook Entry Points

Confirm the exact files and APIs to touch:

• HSA tools hook.
• Code-object reader registry.
• DBI Instrumentor.
• Code-object patcher.
• Instruction decoder and metadata helpers.

Output: short notes on the patch path and any immediate blockers.

2. Add DBI Mode Configuration

Add env parsing for an inactive DBI mode:

• RJ_DBI_SUPERCOLLIDER=1
• RJ_DBI_LOG=1
• RJ_DBI_SC_DELAY=N, interpreted as the number of emitted NOP delay instructions
• optional RJ_DBI_FAIL_CLOSED=1

Output: hook logs that DBI mode is enabled, but still passes original code objects through.

3. Wire HSA Hook To A DBI Stub

Route memory-backed code-object bytes from the HSA load wrapper into a try_patch_supercollider_dbi(...) stub.

Output: the stub receives bytes and returns "unchanged" cleanly.

4. Build And Smoke The Hook

Build the hook and run a tiny HIP or HSA test with:

HSA_TOOLS_LIB=/path/to/librocjitsu_dbi_hooks.so RJ_DBI_SUPERCOLLIDER=1 ./test

Output: the app still runs, and logs show the code-object load path.

5. Parse Code Objects And List Kernels

Construct AmdGpuCodeObject from recorded ELF bytes.

Output:

• target ISA, expected to remain RDNA4 / gfx1201 for this MVP,
• kernel names,
• text ranges,
• per-kernel code sizes.

6. Decode And Count LDS Instructions

Walk decoded instructions and count:

• ds_read*,
• ds_write*,
• ds_*atomic,
• barriers and fence-like instructions.

Output: no mutation yet, just per-kernel LDS statistics.

7. Add Preflight Rejection

Reject or skip kernels with unsupported instructions.

Output: diagnostics that explain why each kernel is patched or skipped.

8. Patch One Known Anchor With Existing NOP Path

Use the current DBI trampoline/instrumentor path to insert one inline nop at a known safe anchor.

Output: live HSA hook patching is proven without SuperCollider logic.

9. Make The Patch Observable

Add a controlled sabotage mode for a known test:

• trap,
• early end,
• or another obvious behavior.

Output: prove the running app executes patched code, not the original code.

10. Implement Minimal ds_read_b32 Check

Patch one supported LDS read form:

• original read,
• delay,
• duplicate read,
• compare,
• trap or report on mismatch.

Output: one selected ds_read_b32 site can be instrumented.

11. Implement Minimal ds_write_b32 Check

Patch one supported LDS write form:

• original write,
• delay,
• duplicate read,
• compare against written value,
• trap or report on mismatch.

Output: one selected ds_write_b32 site can be instrumented.

12. Add A Tiny Racy LDS Test

Create or use an out-of-tree tiny test first:

• one clean missing-race variant,
• one missing-barrier racy variant.

Output: clean variant does not report in ordinary runs; racy variant reports or traps at least sometimes.

13. Generalize To b64 And b128

Extend instruction handling to common wider LDS forms.

Output: DBI can instrument the common LDS widths used by matmul-like kernels.

14. Add Delay Controls

Support:

• fixed NOP-count delay for debugging,
• configurable NOP-count delay through RJ_DBI_SC_DELAY,
• randomized delay when the needed seed values are easy to materialize.

Output: delay can be tuned without rebuilding.

15. Add Report Counter Runtime

If trap mode is working, add a device-visible report counter.

Output: tests can finish and report a count instead of always trapping.

16. Run hip-moi Smoke Tests

Run a narrow hip-moi subset first, then broaden. This is the innermost external iteration corpus for the MVP.

Output:

• patched kernels,
• skipped kernels,
• unsupported opcodes,
• report counts or traps.

17. Run IREE Smoke Tests

Run a narrow IREE ctest -R subset periodically, roughly once per 30-minute session, after the hip-moi loop has produced useful signal.

Output:

• patched kernels,
• skipped kernels,
• unsupported opcodes,
• report counts or traps.

18. Add Positive And Negative Tests

Add a known-clean and known-racy test pair to rocJITsu or the chosen test location.

Output: a repeatable local regression target for DBI SuperCollider.

19. Stabilize Diagnostics

Emit consistent logs such as:

dbi-supercollider: patched kernel=... ds_reads=... ds_writes=... skipped=...
dbi-supercollider: unsupported kernel=... reason=...
dbi-supercollider: reports=...

Output: humans and later automation can tell what coverage was actually provided.

20. Document Usage

Add a short usage note with:

• env vars,
• example hip-moi command,
• example IREE command,
• limitations,
• unsupported instruction policy.

Output: someone else can run the MVP without reverse engineering the hook.

Six-Hour First-Light Schedule

Hour 1:

• Sessions 1 and 2.

Hour 2:

• Sessions 3 and 4.

Hour 3:

• Sessions 5 and 6.

Hour 4:

• Sessions 7 and 8.

Hour 5:

• Session 9 and begin session 10.

Hour 6:

• Finish session 10 or 11.
• Run the tiny racy LDS test.

Expected result: one live code object is patched and at least one selected LDS site can produce an observable report or trap.

Two-Day MVP Schedule

Day 1:

• Establish HSA hook path.
• Log and preflight code objects.
• Prove live patching.
• Instrument one ds_read_b32 or ds_write_b32 site.
• Create or run a tiny racy LDS test.

Day 2:

• Generalize common LDS widths.
• Add report counter if trap mode was first.
• Run selected hip-moi tests.
• Run selected IREE tests as periodic broader coverage.
• Stabilize diagnostics.
• Document usage and limitations.

Initial Test Commands

hip-moi shape:

HSA_TOOLS_LIB=/path/to/librocjitsu_dbi_hooks.so \
RJ_DBI_SUPERCOLLIDER=1 \
RJ_DBI_LOG=1 \
RJ_DBI_SC_DELAY=32 \
ctest --test-dir ~/workspace/hip-moi-build --parallel 8

IREE shape:

HSA_TOOLS_LIB=/path/to/librocjitsu_dbi_hooks.so \
RJ_DBI_SUPERCOLLIDER=1 \
RJ_DBI_LOG=1 \
RJ_DBI_SC_DELAY=32 \
ctest -R '<small-iree-pattern>' --test-dir ~/workspace/iree-build --parallel 8

Start narrow, and keep GPU test fanout near 8. Avoid broad IREE test runs until the unsupported-path diagnostics are trustworthy.

Key Risks

• HSA tools path may not see all code-object creation paths needed by HIP/IREE.
• DBI may need temporary registers or spill handling sooner than expected.
• Because this MVP is instrumentation-only on RDNA4 / gfx1201, any accidental DBT-style architecture translation would be a bug, not a feature.
• Branch/trampoline layout may be harder for dense kernels than for unit tests.
• SuperCollider value checks miss same-value writes and unlucky schedules.
• Trap-first reporting proves instrumentation but is too disruptive for broad test suites.
• Without clear preflight diagnostics, users may overestimate coverage.

Why This MVP Is Worth Doing First

This path exercises the DBI building blocks that later sanitizer modes need:

• code-object interception,
• final-ISA decoding,
• LDS instruction classification,
• live binary patching,
• trampoline/probe emission,
• runtime report plumbing,
• skipped/unsupported diagnostics.

At the same time, it avoids the hard parts of hip-moi/Loom-style "proper" instrumentation until the basic DBI substrate is real.