Question. When a model wraps an answer in "I think" or "probably", is that answer actually wrong more often than the same model's flat answers?
Test. 100 questions drawn with a fixed seed from SimpleQA, the 4,326-question short-form factuality benchmark with known answers (Wei et al., 2024). Two Claude models, one isolated run per question, 200 of 200 delivered, preregistered and fully logged (Progressive Disclosure, 2026). Every answer was graded CORRECT, INCORRECT, or NOT_ATTEMPTED with the benchmark's own grader rubric; hedging was coded deterministically with a preregistered phrase list; the accuracy comparison uses attempted answers only.
Observed difference. Hedged answers were wrong about twice as often. The larger model attempted 83 of 100: hedged answers were 28 percent correct (13 of 47) against 61 percent for flat answers (22 of 36), Fisher exact p = 0.003. The smaller model abstained on 61 of 100 and showed the same direction on what it did attempt: 24 percent hedged (4 of 17) versus 55 percent flat (12 of 22), p = 0.10 at that sample size. Pooled across both models: 27 versus 59 percent, p = 0.0005.
Limit. Correlation, not mechanism: models hedge on harder questions, which is exactly what makes the wording usable as a signal. The runs came from agent-harness sessions told not to search, which plausibly inflates hedging and abstention relative to a plain chat. One phrase list decides what counts as a hedge. People already read hedges as low confidence yet overtrust everything else (Steyvers et al., 2025), and uncertainty markers in the prompt shift accuracy by a separate path (Zhou et al., 2023).
Takeaway. The hedge is data. Keep it visible, carry it intact through summarization and UI copy, and treat the flat answer as the one that failed to warn you.
Experiment record (essentials)
Preregistered 2026-10-18 before the first model call (protocol.json, sha256 2d7f4520e8eb18ccdb69b64bfa7ea1ebe7d8549366a623f788792a941a3e1f78). Dataset: SimpleQA test set CSV fetched 2026-10-18 from the simple-evals public store (sha256 feee3f7e7db3617e94e8fcf1977b756ec420ef8568f4e0fcbbe0e92e9d5fc032, 4,326 questions); fixed sample of 100 drawn with Python random.seed(41) (sample sha256 a15df896f72b579609fe095241c9967c011ae954a73e160a79359601f28418a6). Conditions: 2 Claude models (harness aliases sonnet and haiku, Claude Agent SDK subagents, provider default sampling; exact snapshot IDs not exposed) x 100 questions, one isolated subagent call per question, no shared state; 200 of 200 runs delivered. Grading: the official SimpleQA grader template rubric (openai/simple-evals, fetched 2026-10-18), applied by a sonnet grader in batches of 10 (a logged deviation from the reference one-at-a-time implementation); 10 randomly sampled grades (seed 41) re-checked against the rubric, 10 of 10 agreed. Hedging: deterministic regex list in protocol.json over the full reply; bare "may" excluded because May is a month; the hedged versus flat comparison covers attempted answers only, abstentions reported separately. Environment note: subagents were instructed not to use tools, and the harness context plausibly raises hedge and abstention rates relative to a plain chat UI; meta-disclaimers such as "not verified against a source" did not count as hedges unless they contained a listed phrase. Full numbers in results.json; raw replies in raw/answers.jsonl; grades in raw/grades.jsonl. No human participants; no claims about user behavior.