Firecrawl call brief — Cardmarket scale test status, no-data/stealth question, asks

firecrawl-call-brief.md

Firecrawl call brief — Cardmarket at scale

Status: Firecrawl is our front-runner. Only provider doing true 1-credit single-call pagination at 10k scale, and the lowest projected cost. One open issue to close on this call (the no-data/stealth question) + lock pricing.

---

The numbers (from our real 10k run)

• 9,895 products, concurrency 80, 90.2% success.
• Pagination = exactly 1 credit/product regardless of depth (mean 2.66 pages/product; the actions bundling works — verified, 9,669 credits for 9,669 billed requests).
• ~$0.00065 per useful product → ≈ $4,900/mo @ 7.5M (at ~$0.0006/credit Scale rate — to be firmed up).
• 16.7% of products hit the 300-offer cap (we handle those via filtered follow-up URLs — our side).
• Account already upgraded: 50k credits, 100 concurrent browsers, Enterprise rate limits.

⚠️ The ONE issue to solve on the call: "no-data but billed"

• Of 9,895: 8,930 ok · 755 no-data · 210 fc-500.
• fc-500 (210) = their error → free, retried. Fine.
• no-data (755) = HTTP 200 with NO usable offers, and 739 were BILLED → ~7.6% billed waste.
• Likely a soft-block that basic proxy (1cr) misses. Does stealth (5cr) fix it? If yes, cost 5×'s — so the real trade is success rate vs 5× cost. This is the decision.

📎 Bringing to the call

FAILED_for_firecrawl.csv — 974 failed IDs with firecrawl_status, cardmarket_status, billed flag, retries, timing. Ready to share so they can root-cause.

---

Ask them (in priority order)

1. no-data root cause — we'll send 974 IDs; why do 200s come back with no offers? Soft-block, CF, cache, render timing?
2. Basic vs stealth — does stealth (5cr) actually lift success above ~90%, or is basic enough with a tweak? This is the swing factor on cost.
3. Don't bill no-data — can 200-but-empty pages be treated as failures (not billed)? (~7.6% of spend today.)
4. Enterprise pricing — real $/credit at 7.5M/mo (and whether it's per-credit or committed volume). Confirms the ~$4.9k/mo — or better.
5. Success-rate SLA — what do they guarantee, and how measured?
6. Freshness — confirm maxAge:0 (no stale cached prices) carries no surcharge.
7. Concurrency headroom — 100 browsers enough for 200–250k/day? Room to burst?

Our decision hinges on

success rate × (basic vs stealth cost). If basic holds ~90%+ and they stop billing no-data → clear winner at ~$4.9k/mo. If they need stealth (5×) to hit high-90s → re-run the math vs Crawlbase.

Leverage

We're actively benchmarking Crawlbase (Enterprise Crawler, 500k-URL test) and Bright Data. Firecrawl leads on the bundled-pagination cost model — keep it there.