e2e.md

This is a perfect candidate for a Go "Golden File" test runner. The current shell script is a "Swiss Army Knife" of dependencies (jq, rg, sed, grep, sha256sum, find) that makes your CI environment fragile.

In Go, you can use the os/exec package combined with testify/assert or simple comparison logic to create a much more robust runner.

The "Go E2E Runner" Refactor

Create a file at internal/app/e2e_test.go. This approach uses Go's subtests to run every directory in testdata/e2e as an independent test case.

package app_test

import (
	"bytes"
	"encoding/json"
	"fmt"
	"os"
	"os/exec"
	"path/filepath"
	"strings"
	"testing"
)

func TestEndToEnd(t *testing.T) {
	binary := os.Getenv("STAVE_BIN")
	if binary == "" {
		binary = "../../stave" // Default relative path
	}

	absPath, _ := filepath.Abs(binary)
	if _, err := os.Stat(absPath); err != nil {
		t.Fatalf("stave binary not found at %s. Run 'make build' first.", absPath)
	}

	testCases, _ := filepath.Glob("../../testdata/e2e/e2e-*")
	for _, tc := range testCases {
		name := filepath.Base(tc)
		t.Run(name, func(t *testing.T) {
			runTestCase(t, absPath, tc)
		})
	}
}

func runTestCase(t *testing.T, bin, dir string) {
	var stdout, stderr bytes.Buffer
	
	// 1. Construct the Command
	args := []string{"apply", "--controls", filepath.Join(dir, "controls"), "--observations", filepath.Join(dir, "observations"), "--now", "2026-01-11T00:00:00Z"}
	
	// Handle custom args if present
	if extra, err := os.ReadFile(filepath.Join(dir, "args.txt")); err == nil {
		extraArgs := strings.Fields(strings.ReplaceAll(string(extra), "$CASE_DIR", dir))
		args = append(args, extraArgs...)
	}

	cmd := exec.Command(bin, args...)
	cmd.Stdout = &stdout
	cmd.Stderr = &stderr

	// 2. Execute
	err := cmd.Run()
	exitCode := cmd.ProcessState.ExitCode()

	// 3. Assertions (Defining errors out of existence)
	
	// Check Exit Code
	if expected, err := os.ReadFile(filepath.Join(dir, "expected.exit")); err == nil {
		want := strings.TrimSpace(string(expected))
		if fmt.Sprintf("%d", exitCode) != want {
			t.Errorf("exit code mismatch: got %d, want %s", exitCode, want)
		}
	}

	// Check Summary JSON (The 'Intent' over 'Bytes' approach)
	if expected, err := os.ReadFile(filepath.Join(dir, "expected.summary.json")); err == nil {
		var actualMap map[string]any
		json.Unmarshal(stdout.Bytes(), &actualMap)
		actualSummary, _ := json.Marshal(actualMap["summary"])
		
		// Use a helper to compare normalized JSON
		assertJSONMatch(t, expected, actualSummary)
	}

	// Check Stderr Pattern
	if expected, err := os.ReadFile(filepath.Join(dir, "expected.err.txt")); err == nil {
		pattern := strings.TrimSpace(string(expected))
		if !strings.Contains(stderr.String(), pattern) {
			t.Errorf("stderr missing expected pattern: %s", pattern)
		}
	}
}

func assertJSONMatch(t *testing.T, expected []byte, actual []byte) {
    // Logic to unmarshal both and compare using reflect.DeepEqual 
    // or a library like 'testify/assert'
}

Why this is better for your CLI project:

1. Eliminate Tool Dependencies: You no longer need jq installed on the developer's machine or CI runner. Go's encoding/json handles the comparison natively.
2. Cross-Platform: Shell scripts fail on Windows (unless using WSL/GitBash). This Go test runs natively everywhere.
3. Clean "Stale" Artifacts: Go's t.TempDir() can be used to handle the outdir creation, ensuring that one test run doesn't pollute the next.
4. Debugging: If a test fails, you get the line number in Go and the exact diff in your IDE, rather than just a FAIL string in the terminal.
5. Standard Tooling: You can use go test -v -run TestEndToEnd/e2e-s3 to debug just one specific case.

Recommendation: Use "Golden Files"

Instead of having 8 different files like expected.findings.count, expected.summary.json, etc., consider moving to a Single Golden File per test case that represents the "Perfect Output."

If the output changes intentionally, you can run the tests with a -update flag to refresh the golden files, a pattern common in Go's own compiler tests.

Verdict: Delete the script. Rewriting this in Go makes your test suite part of the compiled binary ecosystem, reduces CI setup time, and makes the "Expected vs Actual" logic much easier to read and maintain.