Who has an incentive to find errors?
Protocol summary · discussion draft · 9 October 2026. Not frozen, not a registered protocol, and no results reported. Design choices below require agreement before data collection.
Claim and comparison
Do rewards for verified error discovery improve sustained, useful discovery after verification costs and false alarms are counted?
Proposed pilot
Randomly assign equivalent audit tasks with or without a reward for independently verified findings. Use a fixed mix of seeded and clean tasks; keep adjudicators blind to incentive assignment. Separate discovery, verification and reward decisions. Repeat rounds to reveal adaptation and gaming.
Measures and failure condition
Measure verified errors per reviewer-hour, missed errors, false alarms, review cost and performance over time. If higher raw report counts disappear after verification costs, or fabrication and low-value findings dominate, the reward design does not support the claim.
Before recruitment
Freeze reward amounts, task difficulty, sample size, minimum net benefit, adjudication rules and stopping thresholds. Disclose who funds and controls verification; include a challenge path for disputed findings.
Control, challenge and stopping
Organizers choose tasks and control records, so their choices can bias the result. Preserve disagreements and protocol changes. Unresolved consent problems, data leakage or compromised scoring should pause the pilot. A negative result must remain reportable. Independent review arrangements and a successor operator are still to be agreed.
Challenge this protocol