For each feature: (1) Why are we doing this? (2) How would we know it worked? (3) What to measure, at what cadence? Then: does the proposed dashboard have a metric that would capture this?
| # | Feature | Why (business value) | How we'd know it worked | What to measure / cadence | Dashboard metric? |
|---|---|---|---|---|---|
| 1 | ABC Pricing Test (analytics implementation) | Find optimal price point among 3 variants | Winning variant has higher revenue per visitor | PostHog: revenue_per_visitor by variant / weekly | PARTIAL — MRR tracked (Q1), but no per-variant breakout in dashboard |
| 2 | Pro Plan Infrastructure (search badge, map pins, photo UI, admin, payment) | New revenue tier for power listers | Pro plan adoptions > 0, Pro MRR contribution measurable | Stripe: Pro subscriptions / weekly | NO — no Pro-specific metric in dashboard_metrics |
| 3 | Coupon Query Hardening | Coupons were silently failing at checkout | Coupon redemptions increase (valid coupons work) | Stripe: invoices with discounts / monthly | YES — coupon-redemptions (Q1) |
| 4 | Server-Side Purchase Tracking | Revenue events weren't firing to PostHog | purchase_completed events in PostHog > 0 | PostHog: purchase_completed count / weekly | NO — no purchase event metric in dashboard |
| 5 | Listing Analytics Page | Listers see how their listing performs | Lister engagement with analytics page | PostHog: listing_analytics_viewed / monthly | NO — no lister engagement metric |
| # | Feature | Why | How we'd know | What to measure / cadence | Dashboard metric? |
|---|---|---|---|---|---|
| 6 | FAQ Cancel Instructions Fix | Wrong cancel path in FAQ frustrated users | Cancel-related support tickets decrease | Manual: ticket count / monthly | YES — no-msg-churn-rate (Q2) captures churn driver |
| 7 | Incomplete Account Messaging Fix | Abandoned-signup users blocked from messaging | Recovered accounts send messages | SQL: messages from late-profile-complete users / weekly | YES — messages-week (Q3) captures the volume impact |
| 8 | Display Name in Emails | Privacy violation: full legal names exposed | All emails use display_identity | Postmark: sample check / weekly | NO — no email privacy metric in dashboard (it's a feature-level metric only) |
| # | Feature | Why | How we'd know | What to measure / cadence | Dashboard metric? |
|---|---|---|---|---|---|
| 9 | PostHog Messaging Instrumentation | Can't analyze inquiry funnel without events | 11 inquiry funnel events fire in PostHog | PostHog: event counts > 0 / daily initially, then weekly | PARTIAL — searcher-msg-rate (Q3) uses these events, but no direct "events firing" check |
| 10 | Address Autocomplete Mobile Styling | iOS Safari URL bar covers autocomplete dialog | Mobile listing creation doesn't regress | PostHog: listing_created on mobile / weekly | YES — mobile-listing-creates (Q3) |
| 11 | Mobile Address Input Styling | iPhone address input broken (repeat regression) | Mobile listing flow works end-to-end | PostHog: listing_created on iOS / weekly | YES — mobile-listing-creates (Q3) |
| 12 | Messaging Strategy Plan | Listers churn when they don't get inquiries | Messages/week increases, no-message churn decreases | SQL: messages/week + no-msg churn / weekly | YES — messages-week (Q3) + no-msg-churn-rate (Q2) |
| # | Feature | Why | How we'd know | What to measure / cadence | Dashboard metric? |
|---|---|---|---|---|---|
| 13 | Fraud Reduction 4 Phases (auto-block, dispute processing, rate limiting) | Revenue protection from chargebacks | Dispute rate decreases, time-to-block decreases | Stripe: dispute rate; SQL: time-to-block / weekly | YES — dispute-rate (Q4) + users-flagged-week (Q4) |
| 14 | Edu Fraud Prevention | .edu scam listings recurring problem | Zero .edu fraud listings created after deploy | SQL: .edu listings created / daily | YES — fraud-msgs-week (Q4) captures the downstream effect |
| 15 | Block User Enforcement | Blocked users could still log in and appear in search | Zero bypass incidents | Rollbar: BlockUserService errors / daily | NO — feature-level metric only (Rollbar), not in dashboard |
| 16 | Disposable Email Blocking Phase 2 | Throwaway fraud accounts via disposable emails | Disposable email registrations = 0 | SQL: registrations from disposable domains / weekly | NO — no disposable email metric in dashboard |
| 17 | saveForm Ownership Check (IDOR) | Any user could edit another's listing | Zero unauthorized saveForm attempts | Rollbar: 403 responses / daily | NO — security metric, feature-level only |
| 18 | Email PII Removal | Inquiry emails exposed phone numbers, full names | PII fields absent from email templates | Source code audit / one-time | NO — graduated, no ongoing dashboard metric needed |
| 19 | Admin Mark-as-Fraud Fix + Activation Gate | Admins couldn't flag fraud; empty listings went live | Fraud marking works; empty active listings count drops | SQL: empty active listings / weekly | YES — empty-listings (Q5) |
| 20 | Display Name Privacy Fix | Legal names in emails | Emails use display_identity | Postmark sample / weekly | NO — feature-level metric |
| 21 | Webhook Null User Guard | Stripe webhooks crashed on null user | Zero webhook Rollbar errors | Rollbar: webhook errors / daily | NO — infrastructure bugfix, feature-level only |
| 22 | User Creation Race Condition | Duplicate user records on concurrent registrations | Zero duplicate-user Rollbar errors | Rollbar / daily | NO — infrastructure, feature-level |
| # | Feature | Why | How we'd know | What to measure / cadence | Dashboard metric? |
|---|---|---|---|---|---|
| 23 | Free Listing Deferred Activation | Incomplete free listings going live | Zero premature activations (NULL title/desc active) | SQL: count / weekly | PARTIAL — empty-listings (Q5) captures the broader problem, not this specific fix |
| 24 | Free Listing Expiration Fix | Stale free listings never expired | Zero stale free listings past 90 days | SQL: count / weekly | YES — stale-free-listings (Q5) |
| 25 | Subscription Type Guards | is_numeric errors on subscription type | Zero Rollbar errors for is_numeric | Rollbar / daily | NO — infrastructure bugfix, feature-level |
| 26 | Null Plan Guard | Edit listing 500 error for pre-plan listings | Zero 500 errors on listing edit | Rollbar / daily | NO — bugfix, feature-level |
| 27 | City Price Stats | HAVING alias error in city stats cron | City price stats cron runs clean | Rollbar / daily | NO — infrastructure |
| # | Feature | Why | How we'd know | What to measure / cadence | Dashboard metric? |
|---|---|---|---|---|---|
| 28 | Blog Migration (WordPress → Laravel) | Laravel blog enables SEO + programmatic content | Blog impressions ≥ pre-migration baseline | GSC: blog impressions / weekly | YES — blog-impressions-week (Q6) |
| 29 | Blog Fixes + Art-Style Covers | 12 broken internal links, raw FAQ text | Blog 404 exits decrease, time-on-page increases | GSC: blog clicks; Rollbar: blog 404s / weekly | YES — blog-impressions-week (Q6) |
| 30 | Travel Nurse Landing Page | Capture TN search traffic with dedicated page | TN keyword position improves, TN page indexed | GSC: position + impressions / weekly | YES — tn-keyword-position (Q6) |
| 31 | SEO Quick Wins (canonical, meta tags, sitemap rewrite) | Technical SEO foundation for organic growth | CTR improves on target pages, indexed pages increase | GSC: CTR + indexed pages / weekly | YES — indexed-blog-pages (Q6) + backlog CTR metric |
| 32 | Font/Image Preload 404 Fix | Preload resources returning 404 | Zero preload 404s in DevTools | One-time manual check | PARTIAL — homepage-lcp (Q6) captures the performance impact |
| 33 | LCP: Defer GetIP Script | Render-blocking third-party script | LCP improves by ~200-500ms | PSI: LCP / weekly | YES — homepage-lcp (Q6) |
| 34 | LCP: Self-Host Manrope Font | External font request blocking render | LCP improves | PSI: LCP / weekly | YES — homepage-lcp (Q6) |
| 35 | LCP: Remove New Relic + jQuery | 200KB unused JS blocking render | LCP improves, FCP improves | PSI: LCP / weekly | YES — homepage-lcp (Q6) |
| # | Feature | Why | How we'd know | What to measure / cadence | Dashboard metric? |
|---|---|---|---|---|---|
| 36 | ChurnKey Retention Flows | Intercept cancel intent, offer saves | Save rate ≥ 15%, retained MRR > $500/month | ChurnKey API: saves/attempts / weekly | PARTIAL — monthly-churn-rate (Q2) captures the outcome, but not ChurnKey-specific |
| 37 | Meta Tags + JSON-LD | Rich results in Google, better CTR | CTR increases on 6 target pages | GSC: CTR for target pages / weekly | PARTIAL — backlog has "GSC CTR" but not yet seeded |
| 38 | Internal Linking (Hub-and-Spoke) | Distribute page authority to deep pages | Deep pages gain impressions | GSC: impressions on linked pages / weekly | PARTIAL — blog-impressions-week (Q6) is too narrow; need site-wide impressions |
| 39 | City Landing Pages (Top 50) | Programmatic SEO for city-specific traffic | City page impressions > 0 within 30 days | GSC: impressions for /sublets/* / monthly | NO — no city page metric in dashboard |
| 40 | Disposable Email Blocking (enforcement) | Block fraud vector at registration | Zero disposable email signups | SQL: disposable domain registrations / weekly | NO — not in dashboard metrics |
| 41 | ABC Pricing Test Decision | Pick winning variant, implement | Revenue impact quantified, winner deployed | Stripe: MRR change after switch / monthly | YES — mrr-total (Q1) |
| 42 | Pro Plan Launch (staging → production) | New revenue tier | Pro subscriptions > 0, Pro MRR contribution | Stripe: Pro subs + revenue / weekly | NO — no Pro-specific metric |
| 43 | Email Lifecycle Strategy | Re-engage listers before they churn | 30-day retention improves for treated cohort | SQL: cohort retention / monthly | YES — retention-30d (Q2) |
| 44 | Listing Quality Score System | Higher quality listings get more inquiries | Listings with score ≥7 get 2x inquiry rate | SQL: inquiry rate by quality score / monthly | NO — no quality score metric |
| 45 | LCP Optimization (full) | LCP at 14.4s is 5.8x over 2.5s target | LCP ≤ 2.5s on homepage and search | PSI: LCP / weekly | YES — homepage-lcp (Q6) |
| 46 | Dispute Resolution System (Phase 2) | Evidence collection + PDF generation for disputes | Evidence packets auto-generated, dispute win rate improves | Stripe: dispute wins/total / monthly | PARTIAL — dispute-rate (Q4) tracks rate but not win rate |
| 47 | Price Range Context in Listing Form | Listers overprice and churn | Price-adjusted listers churn less | SQL: churn rate for listers who adjusted price / monthly | PARTIAL — monthly-churn-rate (Q2) captures total but not the specific intervention |
| 48 | Blog Content Calendar (2-4/week) | Content flywheel for SEO | Blog traffic grows 10%+ month-over-month | GSC: blog impressions MoM / monthly | YES — blog-impressions-week (Q6) |
| 49 | Medical Student Landing Page | Capture med student search traffic | Page indexed, impressions > 0 within 30d | GSC: impressions for /medical-student-housing / monthly | NO — no landing page metric beyond TN |
| 50 | Messaging Experiments (Day-2 coaching email, search nudges) | Increase messaging volume to reduce churn | Messages/week increases for treated cohort | SQL: messages by cohort / weekly | YES — messages-week (Q3) |
| Coverage | Count | % |
|---|---|---|
| YES — dashboard metric directly captures the outcome | 19 | 38% |
| PARTIAL — dashboard has a related metric but not the specific one needed | 12 | 24% |
| NO — no dashboard metric captures this feature's impact | 19 | 38% |
| Question | YES | PARTIAL | NO | Total features | Coverage |
|---|---|---|---|---|---|
| Q1: Revenue | 1 | 1 | 3 | 5 | 20% YES, 40% partial |
| Q2: Churn | 2 | 2 | 0 | 4 | 50% YES, 100% with partial |
| Q3: Demand | 4 | 1 | 0 | 5 | 80% YES |
| Q4: Fraud | 3 | 1 | 6 | 10 | 30% YES, 40% with partial |
| Q5: Supply | 2 | 1 | 3 | 6 | 33% YES, 50% with partial |
| Q6: Organic | 7 | 2 | 1 | 10 | 70% YES, 90% with partial |
| Infrastructure/cross-cutting | 0 | 0 | 6 | 6 | 0% |
| Roadmap (mixed) | 3 | 4 | 4 | 11 | 27% YES, 64% with partial |
| # | Feature | Why it's missing | Should it be? |
|---|---|---|---|
| 2 | Pro Plan Infrastructure | No Pro-specific revenue metric | YES — add Pro MRR to Q1 |
| 4 | Server-Side Purchase Tracking | No purchase event metric | No — infrastructure for other metrics |
| 5 | Listing Analytics Page | No lister engagement metric | No — nice to have, not core |
| 8 | Display Name in Emails | Privacy fix, no ongoing metric | No — should graduate |
| 15 | Block User Enforcement | Rollbar-only metric | No — feature-level is sufficient |
| 16 | Disposable Email Blocking | No disposable email metric | YES — add to Q4 when enforcement ships |
| 17 | saveForm IDOR Fix | Security fix, no business metric | No — security, feature-level |
| 18 | Email PII Removal | One-time fix | No — should graduate |
| 20 | Display Name Privacy | Same as #8 | No — graduate |
| 21 | Webhook Null User Guard | Infrastructure bugfix | No — infrastructure |
| 22 | User Creation Race Condition | Infrastructure bugfix | No — infrastructure |
| 25 | Subscription Type Guards | Infrastructure bugfix | No — infrastructure |
| 26 | Null Plan Guard | Infrastructure bugfix | No — infrastructure |
| 27 | City Price Stats | Infrastructure bugfix | No — infrastructure |
| 39 | City Landing Pages | No city-page metric | YES — add city page impressions to Q6 |
| 40 | Disposable Email Enforcement | No disposable email metric | YES — add to Q4 |
| 42 | Pro Plan Launch | No Pro-specific metric | YES — add Pro MRR to Q1 |
| 44 | Listing Quality Score | No quality score metric | YES — add to Q5 when quality system ships |
| 49 | Medical Student Landing | No landing page metric | MAYBE — add general "landing page impressions" to Q6 |
| Priority | Metric | Question | Justification |
|---|---|---|---|
| HIGH | Pro Plan MRR contribution | Q1 | Pro Plan is a major strategic initiative with zero measurement |
| HIGH | Disposable email registrations | Q4 | Disposable email blocking is a top fraud priority |
| MEDIUM | City/landing page impressions | Q6 | Programmatic SEO is Phase 3 strategy — 50+ pages planned |
| MEDIUM | Dispute win rate | Q4 | Evidence system (Phase 2) designed to improve this |
| LOW | Listing quality distribution | Q5 | Quality score system hasn't shipped yet |
62% of features map to at least a PARTIAL dashboard metric (YES + PARTIAL). But only 38% have a metric that would let you attribute the change to the specific feature. The dashboard tells you "MRR went up" but not "MRR went up because of the Pro Plan." This is the gap between a reporting dashboard and a decision-making dashboard.
The 3 principles require attributability ("how would we know if this specific thing worked?"). The proposed dashboard captures aggregate outcomes (MRR, churn rate, messages/week) but feature-level attribution still lives in the Launches page. This two-level design (dashboard = aggregate health, launches = feature attribution) is actually sound — the dashboard answers "are we healthy?" and the launches page answers "which feature caused the change?"
Verdict: The two-level structure works. Don't try to make the health dashboard show per-feature attribution — that's what the launches page is for.
Only 1 of 5 revenue features has a direct dashboard metric (coupon redemptions). The Pro Plan — a major strategic initiative — has zero measurement. The ABC Pricing Test has no per-variant metric. Revenue is the most important business question but has the lowest dashboard coverage (20%).
Action: Add Pro Plan MRR as a Q1 sub-metric before Phase 2. Consider adding "MRR by plan tier" as a sparkline-capable metric.
7 of 10 organic growth features map directly to a dashboard metric. This is because GSC and PSI provide clean, automatable metrics that directly measure the business outcome (impressions, clicks, LCP). The organic growth question is the model for how all questions should work.
Lesson: Questions backed by automated, specific data sources (GSC, PSI) have better coverage than questions that depend on complex SQL joins or PostHog funnels. Prioritize simple, automated collection.
Not everything needs a KPI. Infrastructure fixes (webhook guards, race conditions, null checks) protect system stability but don't drive business outcomes. They belong in the Launches page with feature-level Rollbar metrics, not in the founder's morning dashboard.
Lesson: The 3 principles self-select — if you can't answer "why does this matter to the business?" then it doesn't belong in the dashboard. Feature-level monitoring is sufficient.
12 features are PARTIAL — the dashboard has a related metric but not the specific one. Examples:
- Free Listing Deferred Activation →
empty-listingscaptures the broad problem but not this specific fix - ChurnKey →
monthly-churn-ratecaptures the outcome but not ChurnKey's specific contribution
This is inherent in an aggregate dashboard. You can't have 50 metrics (one per feature) without recreating the "makes no sense" problem. The resolution is: aggregate metrics in the dashboard, attribution in the launches page, link between them.
The exercise revealed 5 metrics that should be added before the dashboard ships:
- Pro Plan MRR (Q1) — strategic priority, zero measurement
- Disposable email registrations (Q4) — fraud priority
- City/landing page impressions (Q6) — programmatic SEO
- Dispute win rate (Q4) — evidence system
- Listing quality distribution (Q5) — quality score system
Items 1-2 should be added now. Items 3-5 should be added when their features ship.
The proposed 6-question dashboard with ~30 curated metrics would capture the outcome of 62% of the past month's work and 64% of upcoming roadmap items (YES + PARTIAL). The 38% gap is almost entirely infrastructure bugfixes (12 items) that don't need dashboard-level tracking, plus 5 specific metrics to add (Pro Plan MRR, disposable email registrations, city pages, dispute win rate, listing quality).
After adding the 2 high-priority missing metrics (Pro MRR, disposable email), coverage rises to:
- YES: 21/50 (42%)
- PARTIAL: 12/50 (24%)
- NO: 17/50 (34%) — of which 12 are infrastructure bugfixes that correctly don't need dashboard metrics
Effective coverage (excluding infra bugfixes): 33/38 = 87%
The dashboard design is sound. The two-level structure (aggregate health dashboard + feature-level launches page) correctly separates "are we healthy?" from "which feature caused the change?" — and the 3 principles primarily require the latter, which the launches page already provides.