A study comparing Claude Code, mini-SWE-agent, and OpenCode with the same model on SWE-bench Verified found that minimizing harness prompts and tool schemas reduces costs by up to 3x without lowering accuracy.
A UT Austin study ran nearly 35,000 coding-agent runs on SWE-bench Verified and Terminal-Bench while varying how context is compressed, when it triggers, and how much is removed. On Terminal-Bench with Qwen, policies using about a third of the tokens can take 20% to 80% longer than keeping full context.
AutoCompact uses a judge to correct compaction decisions, then applies supervised fine-tuning and reinforcement learning with task-success rewards. Reported pass rates rise by 9.2 points on SWE-bench Verified and 5.0 points on SWE-PolyBench Verified, including when a 256K window never overflows.
A paper from Microsoft and colleagues reports that the best model answers 59.7% of Taste-Bench questions correctly when choosing the better open direction in long engineering and research tasks. Forks decided by later evidence are much harder, a larger reasoning budget does not raise accuracy, and distilling a teacher’s outcome judgment improves held-out SWE-bench Pro success.