Self-correction helps only when the model can verify

"Check your work" is the most common piece of advice given to language models, and one of the least examined. We ran a controlled study to find out when a second pass actually helps. The short answer: self-correction improves accuracy only on problems where the model can reliably tell right from wrong in the first place.

Setup

We drew 4,000 problems across four domains: arithmetic, multi-step math, code repair, and factual recall. Each problem was answered once, then re-answered after a fixed self-critique prompt. We measured three things per pair: whether the answer changed, whether the change helped, and whether the model's stated confidence in each version tracked its correctness.

Results

On arithmetic and multi-step math, self-correction changed 22 percent of answers and improved accuracy by 4.1 points. Changes were strongly correlated with the model's own step-level confidence, which suggests the model could, in a narrow sense, verify its work.

On factual recall the pattern inverted. Self-correction changed 17 percent of answers but reduced accuracy by 1.8 points. Correct first answers were talked out of existence slightly more often than wrong ones were fixed. The model's confidence ratings in this domain were close to uninformative, which supports the reading that verification ability gates the whole technique.

Code repair sat between the two: accuracy rose 1.9 points, and nearly all of the gain came from problems with a failing test the model could run. When the environment provides the verification signal, self-correction works. When the model must supply the signal itself, it works only where its judgment was already reliable.

Reading

Self-correction is better understood as applied verification than as a general reasoning skill. Prompting a model to check its work borrows against a capability that either exists or does not. This is one reason we invest in explicit verifiers rather than longer self-critique prompts.