Run 20 cases,
Is that enough to draw a conclusion?
"The new prompt looks better" is not the deployment standard. EvalEnough reverses the sample size from the smallest actual improvement you care about, and puts ineffective cases and manual review costs on the table.
Dual-version pass rate sample size
MDE is the smallest improvement you're willing to make to change your launch decision, not the pretty number you'd like to see.
Planning is done using two independent proportions of normal approximations, not replacing exact testing, paired design, multi-factor correction, sequential testing, or statistical professional judgment; low pass rates, small samples, and related samples require separate design.
$1 Eval Plan
Generate a frozen evaluation protocol before launch to avoid changing metrics, samples, or stopping conditions after seeing results.
- Assumptions, MDE, sample size, cost, and stopping rules
- Sample freezing, random allocation, and contamination check
- Structured record templates for pass/fail/invalid
- Multi-dimensional metrics, manual review, and final decision records
Evaluation is defined as success
先核对当前来源与工具边界Check current sources and tool limits first