Every evaluated model improves the seed harness, but how it improves, where it stalls, and what it costs separate the evolvers sharply.
All nine evolvers raise the Overall Score above the 29.7 CodeAct starting point, showing that harness evolution delivers gains across model families. Yet the strongest evolved result, GPT-5.6 Sol at 46.3, still trails the 47.5 human-engineered Artificial Harness.
Models find strong structures within a handful of iterations, then keep editing past the point of benefit. Opus 4.8 and GLM-5.2 achieve the highest AnytimeVal scores (51.4 / 51.0) yet freeze below their own peaks.
Search rises from 11.7 to 46.5, while Office moves only a few points. On General tasks the strongest evolved harnesses reach 59.4 and surpass the human-engineered 56.3 baseline.
Swapping the policy to Qwen3.6-35B-A3B or GLM-5.2 preserves the gains, indicating reusable reasoning and tool-use structure rather than patches for one model's quirks.
Models do not consistently make effective use of their full research budgets. The strongest evolvers tend to invest more steps and iterations to reach better performance, and those longer runs also incur substantially higher inference cost.
CodeAct starts with no fetch, HTML handling or result triage, so most of the gap is capability the evolver must invent.
Opus 4.8 reaches 46.5 against the human-engineered 46.7; GLM-5.2 and GPT-5.6 Sol follow at 45.4 and 44.5.
Evolvers add search and fetch tools plus page cleaning that removes scripts, styles and markup noise while retaining links.
By adding and refining search tools, models can achieve substantially larger score gains in the Search domain.
The best gain is +3.3 and no evolver reaches the human-engineered 43.9.
The seed is already capable; remaining gains demand precise, format-aware document and spreadsheet workflows.
Successful variants separate APEX evidence workflows from GDPval creation, recalculation and rendered-artifact inspection.
Specialised office infrastructure is difficult to discover through general trial and error.
GPT-5.6 Sol and Qwen3.7-Max both reach 59.4, above the human-engineered 56.3.
The gain comes from removing over-steering prompt scaffolding rather than adding more instructions.
Recovery, credential redaction and draft-email protection survive the trim.
General tasks reward adaptable reasoning, which is the axis evolution explores most effectively.
It reaches 49.7, crashes most tasks with a malformed verifier, repairs only to 45.4 and never restores the earlier revision.
It hits 46.5 at iteration 3, then tunes locally and freezes at 42.6 with substantial budget unused.
Kimi tests seven variants after its peak and correctly restores the byte-identical best snapshot.
Models add obvious capabilities quickly, then drift into thresholds and prompt edits without paired failure analysis.
Each rollout receives a public task view with scorer and expected fields removed, plus a fresh workspace.
Tool output is screened for benchmark answer keys while preserving the web access Search requires.
All trajectories are checked for paraphrased or obfuscated leakage; affected MiniMax M3 trials are zeroed.
See how budget ceilings and policy swaps change the result.