marin-community/marin#1723 - 2-way sync via ghpr (https://github.com/runsascoded/ghpr)

marin#1723.md

marin-community/marin#1723 "Workspace" step 2: experiments → marin → levanter

Second step in the "uv workspace migration plan" (#1773):

• Merge Levanter into lib/levanter/ (preserving full Git history)
• Migrate GitHub Actions workflows to monorepo structure
• Builds on: #1690 (workspace initialization)
• Next step: Step 3 (Thalas executor - TODO)

flowchart LR
    experiments[experiments]
    marin[lib/marin]
    levanter[lib/levanter]
    data_browser

    experiments --> marin
    experiments --> levanter
    marin --> levanter

Loading

(data_browser stays independent, not a workspace member)

Changes

• Merge Levanter into lib/levanter/ (preserving full Git history)
• Add Levanter member to workspace root pyproject.toml
• Update Marin's Levanter dependency to use workspace reference
• Migrate GitHub Actions workflows to monorepo structure:
  • Rename Marin workflows with marin- prefix
  • Rename Levanter workflows with levanter- prefix
  • Add "Marin - " / "Levanter - " prefixes to workflow names
  • Add path filters to trigger only on relevant changes
  • Update Levanter workflows for workspace (working-directory, --package levanter --frozen flags)
• Update TPU setup scripts for monorepo structure
• Update pre-commit config to exclude lib/levanter/ from Marin license insertion
• Update ReadTheDocs configs to use --frozen flag

Structure After

marin/
  pyproject.toml        # Workspace root
  experiments/
  .github/workflows/
    marin-*.yaml        # Marin workflows
    levanter-*.yaml     # Levanter workflows
  lib/
    marin/              # Workspace member
      pyproject.toml
      src/marin/
    levanter/           # Workspace member (new)
      pyproject.toml
      src/levanter/
      infra/

Migration Scripts

See workspace-migration scripts for:

• step2/main.sh - Main migration script (3 parts: prepare, merge, workflows)
• step2/migrate_workflows_standalone.sh - Standalone workflow migration
• step2/test.sh - Test harness
• Full documentation

Technical Details

• Uses yaya for YAML transformations with insert_key_between() API (zero workflow conflicts, was 9 before)
• Uses tomlkit for TOML transformations (preserves formatting and comments)
• Hermetic and reproducible migration process
• Optional --lock-ref flag to skip uv.lock regeneration (saves 5-10 minutes)
• Full Levanter Git history preserved in merge