CRITICAL: run ALL tasks listed here, in order. Treat this file as fresh every loop. Follow docs/critical-merging-instructions.md and docs/audit-and-harden.md.
Notes:
- Do not use blacklisted commands from AGENTS.md: rm, sed, mv, git stash, or shell wrappers that invoke them.
- If a project script invokes a blacklisted command, do not run that script; run the safe underlying command directly.
- Do all browser automation headless.
- Skip full eslint and full tsc until the final verification pass. Use focused tests and spot checks while working.
- For spot type checks, use: pnpm exec tsc --noEmit --isolatedModules -p tsconfig.json
- Prefer src/server/subroutines over src/server/actions for new server logic.
- Use getEnvSync for provider credentials so Edge Config env overrides are honored.
- Never run SQL writes or migrations without explicit approval. Read-only SELECT queries are allowed.
- Important: prod at https://dancingdragons.cc will be down until May 17th, so use localhost:3001 or localhost:3002 etc via
pnpm run dev --port=3002or whatever
first task:
git status --short
If dirty, inspect the changes, then git add -A and commit with a clear message before beginning new work.
Then scan for conflict markers and resolve them semantically:
git grep -n -E '<<<<<<<|>>>>>>>' -- src app components tests cloudflare-worker remote || true
Resolve according to docs/critical-merging-instructions.md.
task:
(much of this work may already be done, but we need to audit and harden it, and test it thoroughly and do iterative improvements)
we need to make the search for institutions, credentials and certifications more robust - the current code does work, but we can make improvements to user experience;
right now it works, but there are couple of bad user experiences - sometimes instead of hitting enter they have to click start search (when both should work). Or sometimes they may have to click stop then start, when start should work alone. Also, when first loading, sometimes it just shows the first 20-40 items in alphabetical order which is fine, but just that static peek, but scrolling down should introduce and infinite scroll even on first load before search terms are entered. the stream will have to sort (best effort sort) to alphabetical order even as filtering happens in realtime, so rxjs will be imperative to help out with that.
there is a streaming search which does some fancy things like hold db records in memory in next.js - this is good - but now we want to combine all these things:
- when stream starts on server, we can still hold everything in memory in next.js server so that when searching it's loaded faster (when they filter etc), but make sure this is super robust
- client can hold institutions, certifications, and credentials in memory and in indexedb, and maybe in zustand stores if that helps, zustand is probably the right choice!
- stream institutions, certifications and credentials from supabase too (we have the mirror tables)
so the combined/merged stream (use rxjs heavily/liberally) - should combine (1) stream from next.js, (2) stream from memory/zustand, (3) stream from indexeddb and (4) stream from supabase mirror tables and shared entities table too if that helps (dedupe on id/uuid etc)
the prod site (https://dancingdragons.cc is temporarily down, so use pnpm run dev --port 3007 or pnpm run dev --port 3008 if 3007 is taken etc (with webpack or turbopack, but webpack may work better) to test
zustand can sync with indexedb, so ultimately our rxjs can interface with zustand and zustand can abstract away indexed-db syncing; when searching, let's keep the next.js lambda open for up to 300 seconds (the max)? so that search can reuse the in-memory records in next.js
task:
add a loading gif etc to https://dancingdragons.cc/u/user/assessments
also each assessment needs more questions added to it, for example MBTI needs like 50 more questions, etc, same for enneagram and big5 and update at least 10 total.
and make sure to test each one to make sure the scoring/results calculator for each assessment is correct, we need backend logic like src/server/subroutines/assessment-scoring-routines.ts for this and make sure we store and display results like:
u/user/assessment/mbti u/user/assessment/enneagram u/user/assessment/ocean-big5
and on each of those pages show the user's results for every time they did the assessment, not just the most recent one
task:
Harden all email/domain scraping producer pipelines.
Producer entry points:
- Alex job search: u/alex/crons
- Coach recruiting: u/recruiter/crons
- Benefactor leads: u/benefactor/crons
- DD leads/coachees (b2c leads): u/leads/crons and s/leads/crons
- DD bizdev leads (b2b leads): u/bizdev/leads and u/bizdev/crons
Each producer run should aim for at least 25 domains and 25 emails, with a longer-term daily goal of 500 qualified contacts per role. Make sure we are intelligently (via mdp/rl, etc), interchanging/updating strategies to scrape with playwright, puppeteer, browserless.io, jsdom, cheerio, plain native fetch, etc.
Improve multi-page crawling, pagination, directory traversal, and directory-index extraction. Test against examples like:
https://www.pisd.edu/about-our-district/staff-directory https://www.bentley.edu/directory-index https://transparentnevada.com/directory/
Use existing adapters where possible: fetch/cheerio, jsdom, browserless, puppeteer, playwright, chromium, Google CSE, Serper, Brave, DuckDuckGo, Bing, and direct directory parsing.
Keep pipelines isolated. Do not cross-contaminate Alex, recruiter, benefactor, DD leads, or bizdev tables/outcomes.
task:
Harden the email/domain scraping consumption side.
For each producer pipeline, verify there is a matching independent consumer flow that:
- enriches domains/contacts via Hunter, Apollo, Clay, Lusha, Clearbit, Snov, ZoomInfo, etc.
- matches contacts to ICPs and outreach templates
- sends only allowed/internal test emails while outbound sending is disabled
- throttles duplicate outreach for at least 5 weeks
- records outcomes in cron telemetry and outcome tables
Surface the producer and consumer results in u/alex/crons, u/recruiter/crons, u/benefactor/crons, u/leads/crons, and s/leads/crons.
task:
Harden cron observability dashboards.
Dashboards:
u/admin/crons u/leads/crons s/leads/crons u/alex/crons u/benefactor/crons u/recruiter/crons u/marketing/crons u/hr/crons u/ops/crons u/coach_manager/crons u/qa/crons
Dashboards should use more than cron_job_runs and error_tracking. Include available telemetry from:
cron_job_runs error_tracking cron_alert_state lead_outreach_events email_scrape_tool_runs scrape_session_outcomes scrape_run_outcomes
Email scraping daisy chains should continue until at least 25 emails and 25 domains are collected, or 10 minutes has elapsed, whichever comes first. Daisy-chained routes should make at least one hop where applicable.
task:
Harden provider credential handling.
Whenever an env var is missing, an API key is invalid, or a provider returns 401/403, log:
ddlog.warn('env var missing or api key invalid', { service, envVarName, status, reason, resolver: 'getEnvSync', });
Use getEnvSync for Node runtime credentials. Avoid direct process.env provider reads unless used only as a compatibility fallback.
Pay special attention to Serper, Google CSE, Hunter, Apollo, Lusha, Clearbit, SendGrid, HubSpot, Vercel, OpenAI, Gemini, and Cloudflare.
task:
Run docs/debug-prod-final.md and resolve 1-3 current production issues from the last 90 minutes.
Use read-only queries unless explicit SQL write approval is given.
Prioritize:
- cron zombies
- scraping failures
- 0-result lead generation
- provider 401/403/429 warnings
- pinned-tag failures
- Alex auto-apply 0-result runs
- OpenAI/Gemini quota or auth failures
task:
Run all crons in series.
Use:
src/tools/scripts/run-all-crons-serial.sh
Run against prod only with a real production CRON_SECRET. If prod returns consecutive 401/403 responses, stop and report the auth failure instead of hammering endpoints.
Then run against localhost:3009.
If pnpm run dev --port=3009 invokes blacklisted cleanup commands, do not run the wrapper. Start Next directly with the dev env loaded. Kill the local dev server when finished.
Capture failed endpoints and harden 1-3 meaningful failures.
task:
Audit and harden MDP/RL code.
Requirements:
- non-blocking execution
- use dd-proms.ts patterns where appropriate
- cron-based training/evaluation where appropriate
- no blocking page requests
- clear telemetry for decisions and outcomes
Evaluate whether POMDP patterns are needed for:
- assignments
- scheduling
- coach matching
- coach recruiting
- outreach template A/B testing
- scraping strategy optimization
Only edit code where confidence is above 98%.
task:
For pages/components that render images, create dynamic image-loading tests.
Add or improve both:
tests/playwright image tests tests/puppeteer image tests
The tests should discover pages/components using img or image-loading components, visit the relevant pages headlessly, and verify images load.
Prioritize storage fallback order:
- R2
- GCS
- S3
- Google Drive where already supported
task:
Deeply harden u/user/messaging and u/journal.
Use both Playwright and Puppeteer, headless only.
Messaging tests should cover:
- 3 distinct test users via bypass auth
- 1v1 conversations
- group conversations
- private groups where new users cannot be added
- flexible groups where new users can be added and can see old messages
- conversation search
- permissions
- thread titles based on participant names, with UUID as last resort
- message/thread embeddings
Journal tests should share 1-3 entries with:
[email protected] [email protected]
Also test sharing with a group containing those users.
Add a gear icon to the open thread pane. The menu should include:
- show stats
- notification settings
Stats should show word count per person, average reply time, and words per message.
task:
Improve blog rendering and interactivity.
Focus on Mermaid, charts, xychart-beta, gantt, and collapsible/expandable interactive sections.
Improve five data-driven blogs without losing any content/text.
For:
https://dancingdragons.cc/weblogs/building-trust-cornerstone-leadership-part-1
Ensure the author displays as:
By Dr. Mike Mercer, DEL, MSSL
and links to his coaching profile.
When updating blogs, archive previous versions through archived-entities using db-with-transactions-3.ts.
task:
Implement or plan lead/contact locking.
Pages:
u/leads/locks u/coach/leads/locks
Coaches should be able to lock a contact for 3 weeks after first outreach. If the client signs up during the lock, the locking coach wins that client.
Use an atomic transaction or ledger-style table. Avoid race conditions.
task:
Improve coach recruitment outreach email templates.
The template should invite coaches to join the platform, mention corporate and individual clients, mention fractional internal leadership roles, and link to:
https://dancingdragons.cc/careers https://dancingdragons.cc/pub/submit-job-application
Put the template in the existing correct template location and wire it into the coach recruitment outreach flow.
task:
Harden any/unknown types introduced in the last 10 commits.
Search for patterns like:
let db: any let dbro: any let db: unknown let dbro: unknown ReturnType ReturnType
Prefer concrete local types and existing DB helper types. Follow docs/audit-and-harden.md.
task:
Improve pub/search/coach.
Requirements:
- search panel/searchbar spans the top area above results
- submitted query should not disappear unexpectedly
- RxJS state should be stable
- add search history panel
- store last 30 searches in localStorage or IndexedDB
- add a Supabase-backed search query table if an existing pattern exists
- record which searchbar/UI surface submitted the query
task:
Improve pub/search/coaches.
Keep it separate from pub/search/coach.
Requirements:
- independent SSR/client components
- streaming paginated search
- infinite scroll
- RxJS-heavy interactions
- embeddings/vector/tsvector search
- sorting by useful dimensions such as bio length, declared skills, gender, age, etc.
Do not reuse match-page search APIs unless they are explicitly refactored into a new shared streaming API.
task:
Improve shared entity caching.
Entities:
skills industries job functions tags categories institutions credentials certifications
Priority:
- IndexedDB browser cache
- Supabase mirror tables for public/shared reads
- Redis where useful
- Cloudflare prefetch/warming for high-traffic pages
Focus first on skills and tags/categories because they can exceed 2MB. Industries and job functions should remain small enough for lighter caching.
task:
Audit and harden blog/plugin metadata.
For weblogs that need Mermaid/charts, ensure the weblogs table declares required plugins.
Make xychart-beta and gantt render correctly or replace them with a better supported rendering path.
Ensure previous blog versions are recoverable from archived entities.
task:
Cross-pollinate scraping learnings across role pipelines.
Compare what works and fails across:
u/admin/leads u/leads s/leads u/benefactor u/recruiter/leads u/alex/leads u/bizdev/leads
Share successful scraping strategies without mixing data ownership or destination tables.
task:
Update docs/series-assignment-groups-meetings.md only.
Document how coach assignments, meeting series, cadence changes, reminders, and invoices should work.
Reference:
https://dancingdragons.cc/u/coach/postmortem https://dancingdragons.cc/u/coach/client-cadence-update https://dancingdragons.cc/u/coach/forms/single-session-invoice https://dancingdragons.cc/u/coach/group-invoice
Include diagrams and table relationships for meetings, invoices, groups, group sessions, assignments, and series.
Do not change code for this task.
task:
Update docs/intake-user-onboard-client.md only.
Document client onboarding, coach assignment, series creation, contract signing, Stripe $100 reservation charge, billing/banking/ledger flow, first-session scheduling, reminders, and calendar tracking.
Include diagrams.
Do not change code for this task.
task:
Run docs/debug-prod-final.md and resolve 3-9 issues from the last 10 hours if they are still present.
Include investigation of pinned tag failures like:
Failed to list pinned tags
Resolve root causes where possible. Do not run SQL writes without approval.
task:
Build out bizdev lead generation.
Pages:
u/bizdev u/bizdev/home u/bizdev/dashboard u/bizdev/leads u/bizdev/crons
Bizdev is B2B, not individual consumer leads. Keep it independent from u/leads and s/leads.
Add or plan separate B2B ICP tables/templates where appropriate. The goal is to find businesses that may need 20-100 coaches per quarter and send a limited bizdev outreach sequence.
Ensure components/nav/bizdev-header.tsx has a working More menu.
task:
Audit Cloudflare/BrightData caching and preloading.
Ensure these pages are considered for cache warming/prefetching where safe:
u/leads u/benefactor u/admin u/owner u/coach u/user u/recruiter u/livestreamer u/marketing
Do not cache user-private HTML incorrectly. Respect auth cookies and private data boundaries.
task:
Audit and harden u/recruiter/crons and u/recruiter/candidates.
Verify coach recruiting scraping is producing candidate contacts and the consumption/outreach flow is working.
Pages should be present, full-featured, and mobile-friendly.
task:
Improve benefactor lead templates.
Benefactor sells marketing services to small and medium businesses that do outcalls, repeat service, medium-ticket sales, and trust-heavy work.
Create tailored template families for:
Elijah Gizzarelli:
- handyman service
- locksmithing
- roofing
Ekaterina Muntyan:
- dentists
- photographers
- house painters
Vinayak Pandey:
- pest control
- security staff
- DJs
Templates should support interpolation from website content, Yelp reviews, Google Business pages, Facebook pages, and other public signals.
task:
Make sure u/alex and u/alex/crons show successful scraping and applying telemetry.
Avoid misleading 0s. Surface:
- targets picked
- targets staged
- applications attempted
- forms found
- forms submitted
- skips
- profile/config blockers
- ATS/vendor breakdown
task:
Audit and harden benefactor CRM pages.
Pages:
u/benefactor/emails u/benefactor/contacts u/benefactor/hubspot
Remove placeholder copy like “Contact directory not wired up yet” by implementing the data model or a useful fallback.
Add HubSpot sync buttons to relevant benefactor pages.
task:
Add and harden HubSpot pages for DD leads.
Pages:
u/leads/hubspot s/leads/hubspot
Add HubSpot sync buttons to relevant u/leads and s/leads pages.
task:
Create or improve role cron dashboards.
Pages:
u/marketing/crons u/hr/crons u/ops/crons u/coach_manager/crons u/qa/crons
Each dashboard should be full-featured, database-backed, role-scoped where appropriate, and useful for investigating what is working and failing.
task:
Audit and harden MCP.
Consolidate orphan duplicate MCP servers into the registered comprehensive server where appropriate.
Harden exposed HTTP MCP routes:
- dedicated MCP auth secret
- CRON_SECRET fallback only behind explicit opt-in
- parameter allowlists
- redacted errors
- rate limiting
- useful tests
task:
Audit Google indexing and service page SEO.
Prefer Search Console sitemap submission and URL Inspection monitoring for normal pages. Use Google Indexing API only for eligible structured-data URLs.
Plan service page overhaul before coding:
neondb/tables/service-locations-tables.ts neondb/tables/services-offered-tables.ts
Slug requirements:
service/life-coaching service/financial-coaching service/financial-coaching/delaware-a1b2c3 service/delaware-a1b2c3
Avoid marketing-looking catch-all slugs like delaware-usa. Use stable nanoid(6) suffixes for territory/country collision avoidance.
Use declarative Neon diff workflow only. Do not apply SQL without approval.
task:
Run focused verification for touched files.
Required before each commit:
pnpm exec prettier --write git diff --check git grep -n -E '<<<<<<<|>>>>>>>' -- src app components tests cloudflare-worker remote || true
Run relevant focused tests for touched behavior.
Do not run full eslint/full tsc until the final pass.
Task:
Discover a new task! IMPORTANT!
the most important task you have - discover something new to work on - a neglected page or feature left hanging, any mock code that should be real, add a new test file or better/new test cases and test coverage. See the last 1000 commits and changelog.md for info. Use the changelog.md file more.
final task:
Use a clear commit message.
Then:
git add -A git commit --no-verify -m "" git fetch
git push origin HEAD:rw1-alex --no-verify || echo 'needs merge' git push origin HEAD:lb1-alex --no-verify || echo 'needs merge' git push origin HEAD:dev --no-verify || echo 'needs merge'
If any push fails, fetch and merge that branch semantically according to docs/critical-merging-instructions.md, then retry.
(all done)