Last active
March 26, 2026 18:19
-
-
Save belisarius222/d5b081c5544e7e517a97c74e600f1a36 to your computer and use it in GitHub Desktop.
VOID Maintenance Cron Prompt #pagedrop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <!DOCTYPE html> | |
| <html><head><meta charset="utf-8"><title>VOID Maintenance Cron Prompt</title> | |
| <style> | |
| body { font-family: system-ui, sans-serif; max-width: 800px; margin: 40px auto; padding: 0 20px; background: #0d1117; color: #c9d1d9; } | |
| h1 { color: #58a6ff; border-bottom: 1px solid #30363d; padding-bottom: 8px; } | |
| h2 { color: #79c0ff; margin-top: 24px; } | |
| pre { background: #161b22; border: 1px solid #30363d; border-radius: 6px; padding: 16px; overflow-x: auto; font-size: 13px; line-height: 1.5; } | |
| .key { background: #1f6feb22; border-left: 3px solid #1f6feb; padding: 8px 12px; margin: 8px 0; border-radius: 0 6px 6px 0; } | |
| .warn { background: #da363422; border-left: 3px solid #da3634; padding: 8px 12px; margin: 8px 0; border-radius: 0 6px 6px 0; } | |
| .new { background: #23863622; border-left: 3px solid #238636; padding: 8px 12px; margin: 8px 0; border-radius: 0 6px 6px 0; } | |
| </style></head><body> | |
| <h1>VOID Maintenance Cron Prompt</h1> | |
| <p>Runs every 10 minutes. Session-only. Auto-expires after 7 days.</p> | |
| <h2>Prompt Summary</h2> | |
| <pre> | |
| VOID system maintenance — keep the ENTIRE recursive loop moving, including experiments. | |
| 1. SYSTEM HEALTH | |
| - Check task queue (pending/completed/failed) | |
| - Runner alive? If dead, restart. | |
| - Conductor alive? If dead, restart. | |
| 2. IF IDLE (0 pending, 0 claimed) → system stuck. | |
| Investigate and fix IMMEDIATELY: | |
| - Runner logs, failed tasks, missing model config task kinds | |
| - Source pipeline broken → trigger tick, check errors | |
| - Approval blocking → auto-approve | |
| - Code changes needed → make them, don't wait | |
| 3. IF VOID PIPELINE COMPLETE (0 pending + directions forecasted + no experiments running): | |
| This is the EXPERIMENT PHASE. Do NOT report idle. Instead: | |
| - Rank directions by expected_value | |
| - Select top 6 for experiments | |
| - Write/reuse experiment scripts | |
| - Deploy to GPU machines (H100, C45, C47) | |
| - Launch experiments in parallel | |
| - Monitor, collect results, promote winner | |
| 4. AUTO-APPROVE stuck tasks | |
| 5. EXPERIMENT MACHINE HEALTH (every check) | |
| SSH with 3 retries. DM Ted if unreachable. | |
| - H100: ubuntu@16.56.2.253 | |
| - C45: ubuntu@198.13.252.91 | |
| - C47: ubuntu@198.13.252.92 | |
| 6. DISK SPACE (every 3rd check) | |
| </pre> | |
| <h2>Key Design Decisions</h2> | |
| <div class="key"><strong>Immediate investigation on idle:</strong> If 0 pending + 0 claimed, debug and fix now.</div> | |
| <div class="new"><strong>Experiment phase is part of the loop:</strong> When VOID finishes forecasting, automatically select top experiments and run them on GPU machines. Never ask permission. Never report "idle, ready for experiments."</div> | |
| <div class="key"><strong>Self-healing:</strong> Add missing model config task kinds, auto-approve, restart processes, make code changes.</div> | |
| <div class="warn"><strong>Telegram alerts for:</strong> Experiment machine unreachable after 3 retries, or system completely stuck.</div> | |
| <h2>Lessons Learned</h2> | |
| <ol> | |
| <li><strong>v1:</strong> Let system sit idle 2 hours reporting "stable" while source→concept pipeline was broken.</li> | |
| <li><strong>v2:</strong> Fixed pipeline bugs but still asked "want me to run experiments?" after VOID finished forecasting. Running experiments IS the next step — do it automatically.</li> | |
| </ol> | |
| </body></html> |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment