Does AREX-2 actually self-improve?

AREX-2 is a fine-tune of Qwen3.8-27B. It claims to be better at one thing: taking a solution that works and making it better, round after round, without losing track. That claim can be measured. Give the model a task with a numeric score, a time budget and a target, keep the scorer out of its reach, and record every round.

This post does that with one small task: a Python TSP heuristic. AREX-2 runs it three times, its base model three times, and GLM-5.3-flash three times as a hosted reference point. Everything runs through QueryMT and an OpenSpec custom schema that keeps the scoring outside the agent’s reach.

TL;DR #

† Best score recorded before the run hung. In both failed runs the model started its own test script without a timeout. The script blocked the session for 55 and 79 minutes until the harness killed it. Neither run reached its final round or wrote results.md. These scores are shown only so the base model’s search can be compared at all. They are not finished results.

• AREX-2 is consistent, not better. Its three best scores lie within 28 points of each other, and all three runs finished cleanly. The base model has the single best local run (10926.55), but two of its three runs failed.
• AREX-2 iterates, as the paper says. It ran 18 and 19 scored rounds in two runs (the base model ran 4, 4 and 13). It restored the best snapshot 24 times and stopped on its own every time.
• No local model reached the target. GLM-5.3-flash reached it in every run within 11 minutes, using roughly an eighth of AREX-2’s reasoning tokens.
• n = 3 per model, and the scorer depends on wall-clock time. Rescoring the saved best solutions on another machine moves individual runs by up to 150 points. Read the differences between the two local models as tendencies, not as rankings.

The paper in one paragraph #

AREX-2: Advancing Self-Improving Agents through Long-Horizon Reflective Tasks (Qian et al., arXiv:2609.38288, September 2026) defines self-improvement as “the ability to iteratively refine a solution at test time”. It splits that ability into two parts: reflection, which produces a better solution than the current one, and long-horizon execution, which keeps the iteration effective over many rounds. The authors argue that both parts are domain-agnostic. They therefore train on synthesised improvement trajectories from two domains with verifiable feedback: machine-learning engineering and algorithmic programming. The resulting agent, built on Qwen3.8-27B, reports 81.8 on MLE-bench Lite and 70.7 on Frontier-CS, transfers to deep-research benchmarks, and “keeps improving as its budget of rounds grows”.

My question is narrower. Does that behaviour survive a 4-bit quantization and a different agent harness, on a small algorithmic task the model has never seen? And is the difference from the base model visible in three runs each?

The task: improve-tsp #

The task lives in a small public repository, querymt/arex2-tsp. On main it holds a stub, tsp.py, whose solve(points) raises an exception, and a scorer, tests/score.py. Every run in this post has its own branch (bench/b1-arex2 … bench/b3-glm53) with the code the model left behind, its rounds.tsv and its results.md. The scorer runs solve on five fixed random instances of 200 points in [0, 1000]², each in its own subprocess, and prints the mean closed-tour length. An instance is charged a penalty of 1 000 000 if solve crashes, returns anything other than a permutation, or takes longer than 2.0 s. The baseline is therefore 1 000 000.

The change’s objective.md states the rules the model is given:

• target 10900 or lower within 40 minutes of wall-clock time;
• Python standard library only; stop your own search at about 1.5 s;
• seed every RNG, “so that rounds are comparable and a ’new best’ is not luck”;
• don’t read the instances’ seeds or coordinates out of tests/score.py, don’t hard-code tours, and don’t special-case the instance size.

tasks.md has two tasks that act as checkpoints:

- [ ] 1.1 Implement `solve` so it returns a valid tour for every instance and verify the
      scorer prints a score below 100000 (no instance charged the penalty)
- [ ] 1.2 Replace it with a constructive heuristic and verify the score drops below 14000

For calibration: the expected optimal tour for 200 uniform random points in this square is about 0.7124·√(n·A) ≈ 10 000 (the Beardwood–Halton–Hammersley estimate, which the models also derived on their own). Nearest neighbour lands around 13 000, plain 2-opt around 11 500. The target of 10900 is roughly 9 % above optimal. In pure Python with 1.5 s per instance, that takes a working local search and a way out of its local optima.

The prompt is the same for every model:

Implement the OpenSpec change improve-tsp. Get the apply instructions with openspec
instructions apply --change improve-tsp and follow them, including the scorer and results.md.

Everything else, including the loop, the stopping rule and what counts as a round, comes from the change’s schema.

Externalising the score: an OpenSpec custom schema #

OpenSpec changes normally follow the spec-driven schema: proposal → specs/design → tasks → apply. The spec-scored schema is a custom schema that ships with QueryMT (openspec/schemas/spec-scored/ in the task repository). It adds two artifacts and replaces the apply instruction with a scored loop:

The metric itself lives in the change’s .openspec.yaml, next to schema:. OpenSpec ignores x- keys, so openspec validate --strict still passes:

schema: spec-scored
created: 2026-11-07
skip_specs: true
x-score:
  command: 'python3 tests/score.py'   # last stdout line = the score
  direction: min
  budget_min: 40
  target: 10900
  stall_rounds: 3                      # rounds without a new best before changing approach
  protected: ["tests/*"]

Three POSIX shell scripts plus a shared lib.sh (≈390 lines in total) do the rest. None of them knows anything about TSP, Python or the model.

score.sh <change> "<note>" is the only way to record a round. It runs x-score.command and takes the last stdout line as the score (FAIL if it isn’t a number). It then:

• snapshots the whole working tree, untracked files included, to refs/spec-scored/<change>/round-<n>, using a throwaway git index so HEAD and the agent’s index are untouched;
• appends a row to rounds.tsv;
• moves refs/spec-scored/<change>/best;
• prints a status block the model sees as the tool result.

For example, from b3-arex2:

round 10: score=10970.81 | no improvement | best=10951.59 (round 7) | budget: 24m of 40m used, 16m left
target: 10900 (min) - not reached
tasks: 2/2 done
restore best: git checkout refs/spec-scored/improve-tsp/best -- . ':(exclude)openspec/changes/improve-tsp'

The restore command excludes the change folder, so going back to the best code doesn’t also roll back rounds.tsv or the ticked tasks.

stop-hook.sh is wired as a QueryMT stop hook. Each time the agent tries to end its turn, the hook decides from the files alone whether it may stop. If not, it returns {"decision":"block", …} with a reason, and an instruction that QueryMT feeds back to the model as context:

protect-hook.sh is a pre_tool_use hook. It denies file-tool writes to the scorer scripts, rounds.tsv, .openspec.yaml and the protected globs. Shell commands are checked heuristically: a command that names a protected path and contains a write operation (>, tee, sed -i, rm, mv, cp, git checkout, …) is denied. The scorer call itself is allowlisted.

The result is that the loop the paper studies is a property of the harness, not of the model. Any model run with this profile gets the same scorer, the same budget clock, the same stall rule and the same refusal to stop early. What differs between models is only what they do inside the loop. That is exactly what I want to compare.

One detail matters later: the 40-minute budget is advisory. score.sh only reports it. The stop hook only allows the agent to stop once the budget is spent; it doesn’t force a stop. A model can keep scoring after minute 40, and one AREX-2 run did. The hard limit is the harness’s session cap of 90 minutes.

Setup #

The AREX-2 profile #

QueryMT agents are defined by TOML profiles. The benchmark generates one profile per model from a shared template, and a check script verifies that everything outside the model and its sampling parameters is identical across profiles. The parts of the AREX-2 profile that matter for this post:

[agent]
provider = "llama_cpp"
model = "mradermacher/AREX-2-GGUF:AREX-2.Q4_K_M.gguf"

[agent.parameters]
n_ctx = 131072
enable_thinking = true
reasoning_effort = "medium"
max_tokens = 24576
temperature = 0.6

[agent.execution]
max_stop_hook_continuations = 200   # default is 1: let the stop hook drive a long loop

[[agent.hooks.stop.hooks]]
type = "command"
command = 'sh "$(git rev-parse --show-toplevel)/openspec/schemas/spec-scored/bin/stop-hook.sh"'

[profile]
id = "bench-arex2"
name = "arex-bench: AREX-2 27B Q4_K_M"
tags = ["arex-bench", "spec-scored", "single-agent"]

[agent]
provider = "llama_cpp"
model = "mradermacher/AREX-2-GGUF:AREX-2.Q4_K_M.gguf"
tools = [
  "edit", "read_tool", "write_file", "multiedit", "replace_symbol",
  "glob", "search_text", "ls", "index", "get_symbol", "get_function", "find_symbol_references",
  "session_history", "skill", "shell", "todowrite", "todoread", "browse",
]

[agent.permissions]            # empty = autonomous; each run works in a throwaway clone

[agent.parameters]
n_ctx = 131072
n_batch = 2048
n_gpu_layers = 99
flash_attention = "enabled"
kv_cache_type_k = "q8_0"
kv_cache_type_v = "q8_0"
text_only = true
enable_thinking = true
reasoning_effort = "medium"
preserve_reasoning = false     # earlier turns' reasoning is not resent
max_tokens = 24576
temperature = 0.6
top_p = 0.95
top_k = 20
presence_penalty = 1.0

[agent.execution]
max_stop_hook_continuations = 200   # default is 1: let the stop hook drive a long loop

[agent.execution.snapshot]
backend = "git"

[agent.execution.compaction]
auto = true

[agent.hooks]
enabled = true

[[agent.hooks.stop]]
[[agent.hooks.stop.hooks]]
type = "command"
command = 'sh "$(git rev-parse --show-toplevel)/openspec/schemas/spec-scored/bin/stop-hook.sh"'

[[agent.hooks.pre_tool_use]]
matcher = "^(edit|multiedit|write_file|replace_symbol|delete_file|shell)$"
[[agent.hooks.pre_tool_use.hooks]]
type = "command"
command = 'sh "$(git rev-parse --show-toplevel)/openspec/schemas/spec-scored/bin/protect-hook.sh"'

[[middleware]]
type = "limits"
max_steps = 600
max_turns = 200

[[middleware]]
type = "context"
warn_at_percent = 75
compact_at_percent = 85

The model is downloaded from Hugging Face by the llama_cpp provider plugin and runs in-process. Nothing outside QueryMT is involved: no separate inference server, no proxy.

What differs between the three configs #

The one deliberate difference is thinking_budget = 8192 for the base model. Without it, Qwen3.8 tended to spend its whole max_tokens on thinking before emitting a tool call (the reason is recorded in the config itself). AREX-2 doesn’t need the cap. That is itself a behavioural difference, but it also means the base model was run with a constraint AREX-2 wasn’t. I come back to this under threats to validity.

Hardware and schedule #

Both local models ran on one host with 2× NVIDIA RTX 6000 Ada (48 GB each) and a 32-thread Xeon Gold 6134, one run at a time. The matrix was three blocks of {AREX-2, Qwen3.8} in shuffled order. GLM-5.3-flash ran a day later from the same host, through the same harness build, against the same repository commit:

Results #

Best score per run #

The two red, hollow bars are not results. In b1-qwen38 and b2-qwen38 the model reached 11026.22 and 11075.51. It then started a Python one-off script through the shell tool, and that script never returned:

• In b1-qwen38 (minute 33.9) the script timed _two_opt_pass and _or_opt_pass in a while True: loop that only exits when neither pass improves. The deadline argument was time.monotonic() + 3600. It was still running 55 minutes later.
• In b2-qwen38 (minute 10.4) it was a plain solve(pts) call on its own random instance. The round-4 code it was testing also exceeds the scorer’s limit: when I rescored that snapshot, every instance was killed at 12 s (“killed: far over the 2.0s limit”). Without the scorer’s subprocess timeout, the call simply never came back. 79 minutes passed.

QueryMT’s shell tool has no per-command timeout, so the session sat idle until the harness cancelled it at 90 minutes. That is a harness gap as much as a model failure; more on that below. AREX-2 also tested a lot outside the recorded rounds (27 direct tests/score.py calls in b1-arex2 alone), but it did so through the scorer, which kills a slow solver after 12 s. Its longest tool call in any run took 8 seconds.

Best-so-far over time #

Three patterns stand out:

1. GLM-5.3-flash is fast and done. It hits the target within 2–10 minutes, scores a final round and stops. It never had a round without improvement: once below 10900, the stop hook lets it finish.
2. AREX-2 gets most of its gain in the first 20 minutes, then plateaus about 50 points short. In b2 and b3 it kept trying for another 20+ minutes (rounds 13–17 and 8–17) with no new best, restoring the best snapshot between attempts. b1 is the exception. It recorded only 7 rounds, but between minute 8 and minute 50 it ran tests/score.py directly 16 times without recording them, and found its last improvement (10942.32) at minute 49.7, after the advisory budget had run out.
3. Qwen3.8’s working run (b3) improves late and in steps. It had two very bad rounds (87855.84 from a sign bug in its 2-opt delta, 802286.92 with 4 of 5 instances penalised) before reaching 10926.55 at minute 35. That is the best local result, and it came from the only base-model run that didn’t hang.

What happened in each round #

Every recorded round falls into one of four outcomes: a new best; a valid score that isn’t better; a penalised score (≥ 200 000, meaning at least one instance crashed or timed out); or the closing final round. The baseline round is left out.

AREX-2’s 19 “no improvement” rounds are the loop working as designed: one hypothesis, one change, score, keep or revert. It reverted almost all of them by restoring the best snapshot (24 restores across its three runs, against 4 for Qwen3.8 and none for GLM, which never needed one). The four final rounds include one extra in b2, where it scored “verify final state” after writing results.md.

Where the tokens and the time went #

The two local models spend tokens in opposite ways. AREX-2 splits them roughly evenly between reasoning and output: about 32–38 k reasoning tokens per run, and as many again written out as code and tool calls. Qwen3.8 thinks 4–6× more than it writes (83 k reasoning vs 16 k output in b3), even with its 8192-token thinking cap per turn. Its b1 run reasoned through 62 k tokens to produce four scored rounds. GLM-5.3-flash uses a fraction of either.

For both local models almost all the wall-clock time is generation. Measured throughput over generation time is about 21–26 tokens/s for AREX-2 and 33–35 tokens/s for Qwen3.8, and 44–52 tokens/s for the z.ai API (reasoning + output tokens over time spent in LLM requests). Running the scorer and the other tools cost only 1–4 minutes per run. So the 40-minute budget is mostly spent on tokens. A model that writes and thinks less gets more rounds per minute. At these speeds AREX-2 has a 40-minute budget of roughly 55–60 k generated tokens. In b2 and b3 it spent them on 17–18 short rounds rather than on a few long deliberations.

Watching it correct itself #

Aggregate numbers don’t show how a model iterates. The reasoning traces do. QueryMT stores every reasoning block in the session database, so each excerpt below can be tied to the round it led to.

Scroll through the steps. The panel types out the excerpt with the correction markers highlighted, then plots the round that followed on that run’s best-so-far chart. Green is a new best, grey no improvement, red a penalty. The dashed vertical line marks when the excerpt was written.

All quotes are verbatim, and a script checks each fragment against the stored text. […] marks text left out. The full blocks run from 1 000 to 35 000 characters.

Four habits recur across all three AREX-2 runs:

• It checks numbers against the spec before acting. It works out that an identity tour scores about 104 000, which is not below 100 000, so it skips it. It counts stalled rounds against stall_rounds, and notices that 10969.91 is above 10900, not below.
• It treats the snapshot refs as its memory. It restores refs/spec-scored/improve-tsp/best after nearly every failed attempt, 24 times in total, so each new hypothesis starts from the best code rather than from the last broken one.
• It notices its own bookkeeping errors, but doesn’t always act on them. In b2, after 9 rounds, it realised it had never ticked the task checkboxes (“I never ticked the checkboxes!”). It only actually ticked them at minute 41.
• Its diagnoses are not always right. In b2 it blamed the round-9 penalty on math.exp overflowing. Rescoring that snapshot shows ValueError: empty range for randrange() on every instance, the same bug it had found and fixed in b1. It reverted anyway, so the wrong diagnosis cost nothing. But the reflection is a heuristic, not an oracle.

The other two models, for contrast #

The base model reflects too, and in b3 it found a real bug by reading its own delta formula. GLM-5.3-flash barely reflects at all: it uses 2–8 k reasoning tokens per run, and once in every run it accepted a checkpoint it hadn’t actually met.

GLM’s slip is worth spelling out. Task 1.1 says “verify the scorer prints a score below 100000”. GLM returned the identity permutation in all three runs, scored 106070.23 every time, and ticked 1.1 anyway. The stop hook can’t catch this. It checks that each ticked task has its own scored round, not that the round met the task’s threshold, because thresholds live in free-text task descriptions. The final result is unaffected: task 1.2 and the target are both far stricter. But it shows exactly the kind of check that the harness takes over from the model, and where it stops.

Failure modes #

Hung tool calls are the most expensive failure. They are also partly the harness’s fault. QueryMT’s shell tool has no per-command timeout. Cancelling a call kills the process group, but nothing triggers the cancellation except the session cap. A 60-second default timeout would have turned both Qwen3.8 failures into an error message and a few more rounds. Adding one changes the task conditions for every model, so it wasn’t done mid-experiment. It is the obvious change for a next round.

Overrunning the advisory budget is the opposite problem. The 40 minutes only enter the loop through score.sh’s status line and the stop hook’s permission to stop. AREX-2 b1 found its best result at minute 49.7. Judged strictly against the 40-minute budget, its best score is 11003.15, not 10942.32. I report the latter because that is what the run recorded, and flag it here.

How much of this is noise? #

The scorer measures wall-clock time, and every solver in this experiment searches until a deadline (about 1.4–1.85 s). A faster or slower machine therefore gets a different number of iterations and can end with a different tour. A seeded RNG keeps the sequence of moves fixed, not how many of them fit before the deadline. AREX-2 noticed this itself in b1 (“the deadline is time-based (1.5s) […] The main non-determinism is in how many iterations fit in 1.5s.”).

To put a number on it, I checked out every run’s refs/spec-scored/improve-tsp/best snapshot and rescored it three times on a different machine (Apple M4 Pro, Python 3.9.6; the runs used Python 3.12 on the Xeon):

Two things follow:

• Within one machine the scorer is fairly stable. Repeats of the same snapshot differ by at most about 16 points. Across machines it isn’t. Qwen3.8 b3 loses 150 points and AREX-2 b2 loses 75, while their order flips: on the M4, AREX-2 b3 (10941) beats Qwen3.8 b3 (11077). The “best local run” is not a robust ranking.
• GLM’s solutions are robust (±6), and all three stay below the target on both machines. Its search results barely depend on CPU speed. That is a property of the code it wrote, and arguably a better engineering outcome than a slightly lower number that depends on the clock.

I also ran AREX-2’s best solution from b3 three times on its own, outside the script above, and got 10935.87, 10951.59 and 10950.49. The originally recorded 10951.59 is right in that range.

Threats to validity #

• n = 3 per model. Medians and ranges are reported, not means with confidence intervals, because three points don’t support them. The AREX-2 vs Qwen3.8 comparison of best scores is within the cross-machine noise shown above. The robust differences are the qualitative ones: completion (3/3 vs 1/3), round count, restore behaviour and token split.
• The thinking_budget asymmetry. The base model ran with an 8192-token thinking cap per turn and AREX-2 without one. The cap was needed to get Qwen3.8 to emit tool calls at all, but it does constrain how long it can reflect. A run with a higher cap (say 16 k) would be the next control.
• Quantization. Both local models are 4-bit GGUF quantizations from different uploaders (mradermacher Q4_K_M vs unsloth’s UD-Q4_K_M dynamic quant). The paper’s numbers are for full-precision weights. Differences in quantization recipe can matter as much as the fine-tune.
• GLM was not interleaved. Its three runs happened back to back a day later, over a network API whose load and serving stack I don’t control.
• No shell timeout. As discussed, the two Qwen3.8 failures are partly a harness property. A harness with a timeout would likely have let both runs continue. I can’t say where they would have ended.
• One task. A 200-point Euclidean TSP in pure Python is squarely inside the paper’s “algorithmic programming” training domain, and it is small. It says nothing about the deep-research transfer the paper also claims.
• Excerpts are selected. I chose the six AREX-2 excerpts to illustrate correction loops. They are verbatim, but not a random sample. A simple count of correction phrases (wait, actually, hmm, bug, wrong, …) gives ~20 per 10 k characters of reasoning for AREX-2 and 18–23 for Qwen3.8, so the base model “sounds” just as reflective. The difference is what each model does next, not how often it says “wait”.

What this says about the paper’s claim #

The paper’s two ingredients are reflection and long-horizon execution. On this task:

• Long-horizon execution: yes, visibly. AREX-2 completed every run, kept a disciplined hypothesis → score → keep/revert loop for 17–18 rounds in two of three runs, restored the best snapshot instead of building on broken code, and never wedged its own session. That is exactly where the base model failed two times out of three.
• Reflection that produces a better solution: only partly. AREX-2’s improvements plateau around 10940–10970, about 50 points short of a target that a stronger hosted model reached in every run. Its later rounds are well-formed experiments that mostly don’t pay off. The ideas are sensible (Or-opt, 3-opt, annealing, more restarts), but in pure Python under 1.5 s the next step needed a faster inner loop, e.g. neighbour lists and don’t-look bits. GLM-5.3-flash got there in b3 with a “don’t-look worklist local search”; none of the 27B runs did.
• “Keeps improving as its budget of rounds grows”: not observed here. In b2 and b3 the last new best came at minutes 19.8 and 17.6. The following 20+ minutes and 10+ rounds produced nothing. b1’s improvement at minute 49.7 is the only late gain.

So the fine-tune buys reliability inside the loop more than raw solution quality. For an unattended agent that is not a small thing. With the same harness and the same budget, the base model failed twice in a way that left nothing usable. AREX-2 always left a scored best snapshot, an honest results.md and a validated change.

The task repository, including the spec-scored schema and one branch per run (bench/b1-arex2 … bench/b3-glm53) with the final code, rounds.tsv and results.md, is at github.com/querymt/arex2-tsp.

Paper: Hongjin Qian, Chaofan Li, Kun Luo, et al. AREX-2: Advancing Self-Improving Agents through Long-Horizon Reflective Tasks. arXiv:2609.38288, 2026. doi:10.48550/arXiv.2609.38288