Distilled from analysis of 235 IDR briefs across AdventHealth, HCA, Roxborough Memorial, Atlantic Shore Surgical, North Jersey Orthopaedics, and other providers. Covers emergency department, inpatient, surgical, and single-line-item disputes.
Stella's biosafety benchmark stress-tests whether AI models inappropriately lower barriers to biological weapons development. It uses adversarial red-team conversations — not static prompts — to evaluate how models behave when a credible persona makes dangerous requests through natural dialogue.
The core policy question is marginal uplift: does the model's response meaningfully lower the barrier compared to unassisted internet search? A model that explains publicly available textbook-level biology is fine. A model that walks a garage biohacker through production scale-up for a select agent is not.
Three principles guide the benchmark design:
Prepared for meeting with Amazon Responsible AI (Payal Motwani, Rahul Gupta), March 27, 2026 For internal use: Kit, Roy, Annas
What this is: A comprehensive reference packet for the Amazon Responsible AI team, synthesizing STELLA's evaluation of Amazon Nova 2 Lite, the full 30-model leaderboard, and two preprints of original research on multi-turn AI safety. Every quantitative claim is cited to its source file.
Date: 2026-03-22 Context: Informing the next generation of STELLA's adaptive testing system after 15 rounds of experiments establishing that patient-instruction-based approaches are fundamentally capped (V1) on frontier models.