Set up basic NES dev environment for AI agent

NES-ON-MACOS-DEV-PROMPT.md

Prompt: bootstrap an NES dev environment with an agent-controlled emulator

You are a coding agent on macOS. Your job is to set up an environment where you can build NES (Nintendo Entertainment System) ROMs in C, run them in an emulator, and drive that emulator from the command line so you can iterate without a human pressing buttons. Aim for the state described below; report back at each milestone with what you did and what is verified.

Target environment

By the end you should have:

1. An emulator running. FCEUX 2.6+ installed and able to load .nes ROMs. It exposes a Lua scripting API; that API is how you will control it programmatically.
2. A C toolchain for the 6502. cc65 (which provides cc65, ca65, ld65, cl65) and a copy of Shiru/clbr's NESLib, a small C-callable helper library that wraps the bits of the NES hardware most games need (palettes, sprites, controllers, PPU, frame waits, optional famitone2 audio).
3. An "agent bridge" between the two. A small program that lets you, from a shell, send commands to the running emulator and get results back: take screenshots, load ROMs, press buttons, advance frames, read and write memory, save and load state. This is the unlock — it means you can change source, rebuild, push the new ROM into the running emulator, drive a sequence of inputs, and capture a screenshot, all without leaving the terminal.

Keep the working tree small and obvious: bridge files at the top level, each ROM project in its own subfolder.

Step 1 — install tooling

Use Homebrew where possible. Install fceux and cc65. Confirm both work by running fceux --help and cc65 --version. FCEUX on macOS does not ship LuaSocket or LuaRocks and statically links Lua, so do not plan on socket-based bridges from inside the emulator's Lua VM.

Step 2 — design the agent bridge

The bridge has two halves:

• A Lua script (agent_bridge.lua) that loads into FCEUX at launch (fceux --loadlua agent_bridge.lua some.nes &). It runs every frame and exposes a small set of methods that wrap the FCEUX Lua API: emu.pause, emu.unpause, emu.softreset, emu.loadrom, emu.framecount, emu.message, memory.readbyte, memory.readbyterange, memory.writebyte, gui.savescreenshotas, joypad.write, plus thin custom helpers for "press buttons for N frames" and "save/load state slot N".
• A CLI (nesctl, Python 3 stdlib only) the user runs from a shell. Subcommands like nesctl screenshot path.png, nesctl loadrom path.nes, nesctl press A 5, nesctl peek 0x0070 16, nesctl pause, nesctl wait 30, plus a generic nesctl call <method> [args...] escape hatch.

For transport: do not use sockets — use a pair of files in /tmp/nesctl/ (one for requests, one for responses), with atomic writes via rename, and an flock on the CLI side to serialise concurrent calls. The Lua script polls the request file every frame; when it appears, it executes the method and writes the response file. Use gui.register(callback) so polling continues even while the emulator is paused. Use a simple line-oriented text protocol (method on first line, one arg per subsequent line; response is OK\n<type-prefixed-value> or ERR\n<message>). Keep one command in flight at a time — for an agent driving a single emulator, you do not need concurrency.

For "press buttons for N frames" and similar effects that need frames to elapse, the command handler cannot call emu.frameadvance() directly because that yields the running script's coroutine. Instead, set a "pending" record in Lua, return a deferred sentinel so no response is written yet, and have the main loop tick the pending record down each frame and write the response when it reaches zero. The CLI blocks reading the response file until then.

Document the wire format and architecture in a top-level Markdown file as you go, so a teammate (or the next agent session) can understand it without reading code.

Step 3 — first ROM, sanity-check the bridge

Write or borrow a tiny "hello world" ROM in 6502 assembly that displays static text on the title screen. Build it with ca65 + ld65, load it with nesctl loadrom, and confirm:

• nesctl screenshot /tmp/test.png produces a PNG showing the title screen.
• nesctl peek 0x0000 16 returns 16 bytes of zero-page RAM.
• nesctl press A 5 returns roughly 80 ms later (one NES frame is ~16.67 ms; 5 frames ≈ 83 ms).

If those three work, the bridge is good.

Step 4 — set up a C-based ROM project

Now move from assembly to C. Make a subfolder for the project. Inside it:

• A lib/ directory containing NESLib's source files: crt0.s, neslib.h, neslib.sinc, display.sinc, famitone2.sinc, lz4vram.s. Get them from the canonical mirror at github.com/clbr/neslib.
• A nes.cfg linker config for NROM-256 (mapper 0, 32 KiB PRG, 8 KiB CHR). The file needs MEMORY regions for ZP, HEADER, PRG, DMC, VECTORS, CHR, RAM; matching SEGMENTS named HEADER, STARTUP, LOWCODE, INIT, ONCE, CODE, RODATA, DATA, VECTORS, SAMPLES, CHARS, BSS, HEAP, ZEROPAGE; and weak SYMBOLS for NES_MAPPER, NES_PRG_BANKS, NES_CHR_BANKS, NES_MIRRORING, __STACKSIZE__. Plus the standard FEATURES block declaring CONDES for constructors, destructors, and interruptors. Use github.com/clbr/nes/tree/master/cnrom/nes.cfg as a reference — it is known good.
• A chr.s containing the 8 KiB CHR ROM, split into a 4 KiB sprite pattern table and a 4 KiB background pattern table, each tile encoded as 16 bytes (8 low bitplane bytes followed by 8 high bitplane bytes).
• A main.c with your game loop. Two things to know up front: NESLib expects a zero-page byte called oam_off declared in C with the #pragma bss-name (push, "ZEROPAGE") dance, otherwise the linker fails on _oam_off; and oam_meta_spr(x, y, sprid, data) takes four args, not three (sprid is the sprite buffer offset, pass 0).
• A Makefile that compiles main.c through cc65 then ca65, assembles chr.s and lib/crt0.s with ca65, and links everything against nes.lib via ld65 -C nes.cfg. Add a reload target that calls ../nesctl loadrom $(abspath $(ROM)) and a shot target that builds, reloads, waits a few frames, and screenshots — that is the inner loop.

Build it, load the resulting .nes in FCEUX, and confirm a sprite renders.