Empirical study of Matt Pocock's /retro skill on Claude Code. Of seven evaluations where the targeted problem actually occurred, two harness changes eliminated it, one reduced it and four showed no reduction.
/retro asks an agent to review a finished session and propose changes to its harness: software, configuration and instructions. Treude and Baltes ran Claude Code on 30 SWE-bench Verified issues, coded the 125 proposals it produced, implemented 24, and tested 12 harness changes on later issues from the same repositories, in fresh sessions with and without each change.
Most proposals pointed at something that really happened in the session, but 18 of 43 lint-related ones recommended a practice regardless of what had happened. Some changes also added work on unrelated issues.
The takeaway applies to any self-improving agent setup, and to every AGENTS.md that only ever grows: generating a change isn't enough. You need a way to check whether it helps on later tasks, and to revise or remove it when it doesn't.