Interactive · Week 11 · Section 2

Three weak verifiers can beat one strong verifier — but only when their errors are independent.

A generator produces 50 outputs, 30% of which are wrong. N verifiers each independently judge accept/reject. The voting rule combines them into the ensemble verdict. Slide N up; watch the catch rate climb. Slide ρ (correlation) up; watch the ensemble collapse to single-verifier performance. The visceral lesson: verifier ensembles are powerful when errors are independent, useless when verifiers all make the same mistakes. Production verifier panels need different model families, prompts, or grounding mechanisms — anything that breaks the correlation structure.

Ensemble configuration

Number of verifiers (N) 3

how many independent verifier calls per output

Voting rule

at least ⌈N/2⌉ verifiers must reject

Verifier characteristics

Per-verifier accuracy (p) 0.75

P(verifier votes correctly) on a single sample

Error correlation (ρ) 0.30

ρ=0 fully independent · ρ=1 verifiers always agree

Catch rate

—

% of wrong answers rejected

False reject rate

—

% of correct answers wrongly rejected

Net accuracy

—

across all 50 samples

vs single verifier

—

accuracy lift from ensembling

50-sample verdict grid · top stripe = truth · middle = verifier votes · bottom = ensemble + outcome

—

Truth correct wrong

Vote accept reject

Outcome caught error clean accept over-reject missed error

Same data · alternative configs

current config has accent border

What's happening here

Sample population: 50 generator outputs, of which 15 (30%) are wrong and 35 (70%) are correct. Each verifier's vote is generated as: with probability ρ·(1−p) all verifiers err in unison (correlated error); with probability (1−ρ)·(1−p) each verifier independently makes its own error. Marginal per-verifier accuracy is preserved at p across all ρ; what changes is the joint structure. Voting rules: OR rejects if ≥1 verifier rejects, AND rejects if all reject, Majority rejects if ⌈N/2⌉+1 verifiers reject (for even N) or ⌈N/2⌉ (for odd N). Source-of-truth lives as <VerifierEnsembleExplorer/> in the curriculum codebase.