Two prompt specs for an interactive React app that turns images into music: one is a detailed engineering spec, the other is a concise, user-level feature brief.

01-image-to-music-visualizer-spec.md

Create an Interactive Image-to-Music Visualizer

I want a single web app in React that turns an image into a grid of colorful blocks, then plays music based on the colors — in real time.

How it should look and feel:

• The top has a title, a Play/Stop button, an image upload, and a toggle to show diagnostics.
• On the left:
  • A waveform that moves with the sound.
  • Below it, a large pixelated “image map” showing the scanned blocks, with a moving highlight showing which block is being played.
• On the right:
  • Sliders and dropdowns for tempo, pitch, scale, wave type, effects, block size, and reverb.
  • A presets menu for styles like Ambient, Techno, Chiptune, LoFi.

How it should work:

• You can upload any image. It gets broken into rows of blocks (not just columns) from left-to-right, top-to-bottom.
• Each block’s average color decides which note plays, how bright or soft it is, and what effects are applied.
• The sound should change instantly when you move any slider — no need to restart.
• The image map stays visible even if the music is paused, and resizing sliders should never make it disappear.
• Presets instantly set all controls for a certain style.
• The waveform and block highlight move in sync with the sound.

Tech notes for the AI:

• Use React with hooks, Tailwind for styling, and the Web Audio API for sound.
• Draw the pixelated image map without distortion, keeping the aspect ratio correct.
• Process blocks in row order.
• Effects include harmonics, filter, tremolo, vibrato, reverb, and bass boost.
• Show diagnostics only when toggled on.

Deliverable:

• A single React component I can paste into a project and run.

02-audio-visual-sonification-spec.md

Build an Audio–Visual Sonification App (replicate this spec exactly)

Create a single React component (functional, hooks) named SonificationApp that I can import and render. It must match the behavior and structure below precisely.

Tech + Structure

• Use React with hooks (useState, useEffect, useRef, useMemo).
• Export default the component.
• Use Tailwind-style utility classes for styling exactly as referenced below (no CSS files).
• No external state managers or audio libs; use Web Audio API directly.
• Component layout: Header, Main (Visualizer + Control Panel), Footer.

UI Layout (must match)

• Header: title “Audio–Visual Sonification”, subtle subtitle “Blocks scan • Rich synthesis”, and 3 controls:
  • Play/Stop button.
  • Upload Image <input type="file" accept="image/*">.
  • Show/Hide Diagnostics toggle button.
• Main area → grid with two sections:
  1. Visualizer card: waveform canvas (top) + Image Map (bottom) with a Height slider that changes the image-map height in pixels.
  2. Control Panel card: controls in grouped sections (Core, Tone, Modulation, Filter & Reverb, Blocks) plus a Presets dropdown at the top.
• Footer: two short helper lines (“Upload an image…”, “Tip: Larger blocks…”).

Visualizer

• Waveform canvas rendering from an AnalyserNode time-domain buffer (line plot).
• Background gradient (#0f172a → #111827), thin grid lines every 24px.
• Runs on requestAnimationFrame.
• Image Map canvas directly below: draws the processed (downsampled) image aspect-fit with image smoothing disabled, plus a cyan stroke rectangle indicating the current block (step) being sonified.
• The marker MUST draw even when paused (based on the last stepRef index).
• The Image Map height slider (120–520px) must update the backing store and repaint immediately even when not playing.

Image Handling & Block Grid

• On upload, load image into an offscreen canvas and downsample preserving aspect to a working size max 256×256.
• Read pixels via getImageData(0,0,W,H) (no external libs).
• Build a row-major grid of blocks using two sliders:
  • Block Width (px): 1–32
  • Block Height (px): 1–32
• For each block, compute average RGB across its pixels, then derive:
  • h from HSV (0–360) for scale degree mapping.
  • l from HSL (0–100) for brightness-driven variations.
• Degree mapping: degree = floor(sqrt(h/360) * (scale.length-1)) (note the sqrt curve).
• Octave mapping: oct = floor((l/100) * 3) - 1 → range -1 to +2.
• Velocity: velocity = clamp(0.3 + (l/100)*0.7, 0.1, 1) → ensures every block is audible.
• Recompute blocks whenever Block Width, Block Height, or Scale changes.
• Traversal order: strictly left-to-right, top-to-bottom (row-major). The sequencer iterates through the precomputed blocksRef in that order.

Audio Engine (Web Audio API)

• Initialize on first Play:
  • AudioContext, AnalyserNode (fftSize 2048, smoothingTimeConstant 0.85)
  • Master gain → ctx.destination
  • Wet/Dry paths with a ConvolverNode for reverb (no IR fetch; build via generated impulse response: stereo noise with exponential decay; duration and decay tied to reverbTime).
  • Bass boost via lowshelf at 200 Hz (dB from slider).
• Live parameter updates during playback (no restart) using a params ref pattern to avoid stale closures.
• Sequencer timing:
  • Steps per beat = speed (0.25–8).
  • Recompute interval timing immediately if BPM or speed changes while playing.
  • Compute step ms = 60000 / bpm / clamp(speed, 0.25, 8), with a floor (≥20ms).

Note Scheduling (per step)

• Map from block:
  • Use scaleIntervals from the selected Scale and the calculated degree and oct.
  • Base note is Hz slider; convert with freqToMidi/midiToFreq helpers to add intervals.
  • Brightness (from HSL l) drives both velocity and filter cutoff tracking.
• Synthesis chain (per note):
  • Oscillator type: sine/square/sawtooth/triangle (select).
  • Optional harmonics:
    • +fifth at 1.5× freq, level via fifthLevel (0–1).
    • +octave at 2× freq, level via octaveLevel (0–1).
  • Vibrato: a dedicated LFO (rate Hz, depth in cents converted to Hz offset and applied to osc.frequency).
  • Filter: low-pass, Q = resonance; cutoff = filterBase + brightness01 * filterTrack.
  • Tremolo: separate LFO (rate Hz, depth 0–1) applied to a gain before bass boost.
  • Bass boost: lowshelf at 200 Hz, gain (dB) from slider.
  • Wet/Dry: split to a Convolver (wet) and a dry path; wet/dry mix from sliders; convolver buffer regenerates when reverbTime changes.
  • ADSR: short attack to peak = (0.15 + 0.85*velocity) * volume^2, decay to 60%, release before note end; total note length = clamp(stepDur * noteMult * (0.6 + 0.8 * velocity), 0.06, 2.0).
• Analyser taps the mixed wet+dry to feed the waveform canvas.

Controls (exact list + behavior)

• Core
  • Volume (0–1, step 0.01) — perceptual scaling via volume^2.
  • BPM (40–240)
  • Speed (0.25–8, step 0.25)
  • Note Length Multiplier (0.2–2, step 0.05)
• Tone
  • Oscillator Type: sine, square, sawtooth, triangle
  • Scale: Major, Minor, Pentatonic, Blues, Chromatic, Whole Tone
  • Base Note (Hz 110–880)
• Harmonics
  • 5th Level (0–1), Octave Level (0–1)
• Modulation
  • Vibrato Rate (Hz 0–12), Vibrato Depth (cents 0–50)
  • Tremolo Rate (Hz 0–20), Tremolo Depth (0–0.95)
• Filter & Reverb
  • Filter Base (Hz 100–4000), Filter Tracking (Hz 0–6000)
  • Resonance Q (0.3–10)
  • Reverb Mix (0–1)
  • Reverb Time (0.2–4 s) → regenerate impulse on change
  • Bass Boost (dB -6 to +18)
• Blocks
  • Block Width (px 1–32), Block Height (px 1–32)
• Image Map
  • Height (120–520 px) — repaints immediately when changed, even if paused.
• Presets dropdown (exact names + values):
  • — (no change)
  • Ambient Pad: oscType: sine, scaleName: Pentatonic, baseNote: 196, bpm: 70, speed: 0.5, noteMult: 1.6, vibratoRate: 4, vibratoDepth: 6, tremoloRate: 2.5, tremoloDepth: 0.15, fifthLevel: 0.25, octaveLevel: 0.35, filterBase: 600, filterTrack: 3000, resonance: 0.8, reverbMix: 0.4, reverbTime: 2.8, bassGain: 4.
  • Chiptune Arp: oscType: square, scaleName: Major, baseNote: 330, bpm: 150, speed: 4, noteMult: 0.35, vibratoRate: 6, vibratoDepth: 4, tremoloRate: 12, tremoloDepth: 0.25, fifthLevel: 0.5, octaveLevel: 0.2, filterBase: 1200, filterTrack: 1500, resonance: 0.9, reverbMix: 0.15, reverbTime: 1.2, bassGain: 2.
  • Techno Seq: oscType: sawtooth, scaleName: Minor, baseNote: 110, bpm: 128, speed: 2, noteMult: 0.5, vibratoRate: 5, vibratoDepth: 3, tremoloRate: 8, tremoloDepth: 0.3, fifthLevel: 0.3, octaveLevel: 0.1, filterBase: 400, filterTrack: 5000, resonance: 1.2, reverbMix: 0.2, reverbTime: 1.5, bassGain: 8.
  • LoFi Keys: oscType: triangle, scaleName: Blues, baseNote: 247, bpm: 84, speed: 1, noteMult: 1.1, vibratoRate: 5, vibratoDepth: 7, tremoloRate: 3.2, tremoloDepth: 0.22, fifthLevel: 0.2, octaveLevel: 0.15, filterBase: 700, filterTrack: 2600, resonance: 0.7, reverbMix: 0.3, reverbTime: 2.2, bassGain: 5.

Behavior Guarantees

• Real-time: Changing ANY control updates the sound immediately on the next scheduled step; BPM/Speed changes must retime the interval instantly.
• drawImageMap(): must exist and be called on:
  • After processing an image
  • On window resize
  • On image map height changes
  • During visualizer loop
  • When paused and grid/scale changes
• When paused, changing Block Width/Height or Scale must repaint the Image Map and keep the marker index stable (no forced reset).
• Diagnostics panel with simple runtime checks (e.g., midiToFreq(69) ≈ 440, existence of convolver/analyser/wet-dry, blocks computed, drawImageMap defined).

Implementation Notes

• Use helper functions: midiToFreq, freqToMidi, clamp, rgbToHsv, rgbToHsl.
• Store image pixel buffer + dimensions in refs; compute blocks into a blocksRef array and keep colsRef/rowsRef.
• Sequencer step index: stepRef, wraps around blocksRef.length (or 64 fallback when no image).
• Use a ref-backed params object (paramsRef.current = {...} in an effect) to prevent stale closures.
• On teardown (Stop): clear requestAnimationFrame and the step interval.

Deliver only the React component source (ES module with import React, { ... } from "react" and export default function SonificationApp() { ... }). Do not include any build config, HTML shell, or external assets.