T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks

01Overview概览

The Linux terminal is the sharpest test of autonomous long-horizon execution: it binds abstract planning to irreversible side effects, and every attempt is adjudicated by the task's own held-out verifier rather than by a preference model.

Linux 终端是对自主长程执行最锋利的检验：它把抽象规划绑定到不可逆的副作用上，而每一次尝试都由任务自带的留出验证器裁决，而不是由偏好模型裁决。

We introduce T1, obtained by post-training Qwen3.5-122B-A10B with PPO on terminal tasks. Two difficulties dominate this regime, and the recipe is organized around them.

我们提出 T1：以 Qwen3.5-122B-A10B 为底座、在终端任务上用 PPO 后训练得到。这个场景由两个困难主导，整套配方就是围绕它们组织的。

Train–inference consistency for sparse models稀疏模型的训推一致性

Experts hold most of the parameters, and a discrete router picks a small subset per token. Tiny numeric differences between the two stacks flip those selections, while the harness perturbs the token stream at every turn boundary. We fix the two axes separately: TITO for tokens, R³ for experts.

专家持有绝大部分参数，而离散路由器为每个 token 只挑出其中一小部分。两套栈之间极小的数值差异就会翻转这些选择，而 harness 又在每个轮次边界扰动 token 流。我们把这两条轴分开解决：TITO 管 token，R³ 管专家。

Dense reward from execution来自执行的稠密奖励

A rollout batch costs hundreds of sandbox-hours, yet a binary outcome returns one bit per trajectory, and our first binary-reward campaign never passed its supervised baseline. We score by the number of passing assertions on a fixed global scale, feeding a warm-started critic.

一个 rollout batch 要花掉数百沙箱小时，而二值结果每条轨迹只回报 1 bit；我们第一次二值奖励实验从未超过监督基线。我们改为按通过断言的个数在固定的全局尺度上计分，并喂给一个 warm-start 过的 critic。

Model size versus Terminal-Bench 2.1 resolved rate — **Figure 1 | Performance vs. model size under the same agent harness.** At 122B, T1 resolves **64.0%** of Terminal-Bench 2.1, above GPT-5.4 (54.8%), DeepSeek-V4-Flash (56.9%) and Claude Opus 4.6 (63.8%), close to Claude Opus 4.7 (66.1%), and best in its size band.

**图 1｜同一 agent harness 下的规模—性能定位。**在 122B 这一档，T1 在 Terminal-Bench 2.1 上解决 **64.0%**，高于 GPT-5.4（54.8%）、DeepSeek-V4-Flash（56.9%）与 Claude Opus 4.6（63.8%），逼近 Claude Opus 4.7（66.1%），是同规模区间里最好的模型。

What this report contributes本报告的贡献

A 122B MoE terminal agent trained purely by RL on executed outcomes, up to 300+ tool-call turns per task.
A stabilization stack for large-scale sparse agentic RL: TITO, routing replay, and a warm-started critic.
A dense execution reward with its measured behaviour, plus the corpus audit that makes such a reward safe to optimize.

一个 122B MoE 终端 Agent，纯粹用执行结果上的强化学习训练，单任务可达 300+ 次工具调用。
一套面向大规模稀疏 Agent RL 的稳定化栈：TITO、路由回放，以及 warm-start 的 critic。
一个稠密执行奖励及其实测行为，以及让这种奖励可被安全优化的语料审计。

02Approach方法

A training backend and inference replicas run concurrently on disjoint accelerators under slime, pipelining step t training with step t+1 generation. Our terminal-agent integration attaches additively through public extension points rather than by forking the framework.

训练后端与推理副本在互不相交的加速器上并发运行（框架为 slime），把第 t 步的训练与第 t+1 步的生成流水化。我们的终端 Agent 集成通过公开扩展点加法式接入，而不是 fork 框架。

**Figure 2 | The T1 training pipeline.** **Left:** each task is self-contained, and its verifier reports every assertion individually, which makes the reward executable rather than modelled. **Middle:** inference replicas serve the behaviour policy under oversampling; the assembler turns interaction logs into TITO-stitched samples carrying routing records and the dense reward; the trainer updates critic then actor before publishing weights. **Right:** sandboxes are created, driven turn by turn, verified and reclaimed.

**图 2｜T1 训练流水线。**左：每个任务都是自包含的，其验证器逐条报告每一个断言，这让奖励是被执行出来的而非被建模的。中：推理副本在过采样下提供行为策略；装配器把交互日志变成 TITO 拼接的训练样本，并携带路由记录与稠密奖励；训练器先更新 critic 再更新 actor，然后发布权重。右：沙箱被创建、逐轮驱动、验证并回收。

Three ingredients. Select one to open it.

三个要素。点击其中一个展开。

Recursive synthesis of terminal training tasks

No reinforcement signal can be richer than what its verifier can measure, which makes task construction a design decision rather than a preprocessing step. Based on RST, each task is self-contained: resource limits, a long-horizon instruction, an environment specification, a held-out verifier, and a reference solution the agent never sees.

The verifier is the load-bearing part, because it reports every assertion separately instead of one pass or fail. T1 trains on a selected high-quality 15K subset of RST, chosen along verifier, solution, instruction and task-value dimensions.

没有任何强化信号能比它的验证器所能测量的更丰富，这让任务构造成为一个设计决策，而不是预处理步骤。基于 RST，每个任务都是自包含的：资源上限、一条长程指令、一份环境规格、一个留出验证器，以及 Agent 永远看不到的参考解。

验证器是承重结构，因为它分别报告每一条断言，而不是给一个总的通过或失败。 T1 在从 RST 中挑出的高质量 15K 子集上训练，挑选依据是验证器、解法、指令与任务价值四个维度。

TMax-15k — tasks converted from a public corpus. Its verifiers emit only a binary outcome, so only binary rewards are possible here.
RST-38k — synthesized tasks: RST iteratively extends seed solutions, realigns verifiers and instructions, and sandboxes each task before recursive seeding.
T1-15k — the audited pool whose verifiers report per-assertion outcomes. It is materialized in quality-rank order, so per-epoch shuffling is mandatory.

TMax-15k：由公开语料转换而来的任务。它的验证器只发出二值结果，所以这里只能做二值奖励。
RST-38k：合成任务。RST 迭代地扩展种子解法、重新对齐验证器与指令、把每个任务沙箱化，然后递归再播种。
T1-15k：经过审计的数据池，其验证器逐条报告断言结果。它按质量排名顺序物化，因此逐 epoch 打乱是必须的。

**Figure 3 | Category composition of T1-15k.** Angle encodes share exactly, while radius is a rank-based scale that keeps small slices visible. The pool is concentrated in command-line engineering work, with the top five categories accounting for 67.9% of all tasks.

**图 3｜T1-15k 的类别构成。**角度精确编码占比，而半径是基于排名的刻度，用来让小切片仍然可见。数据池明显集中在命令行工程工作，前五类占全部任务的 67.9%。

Dataset construction and audit数据集构造与审计

**Figure 4 | The eight audit dimensions and their weights.** Aggregated by facet: Verifier **45%**, Solution **25%**, Instruction **20%**, Task Value **10%**.

**图 4｜八个审计维度及其权重。**按面归并：验证器 **45%**、解法 **25%**、指令 **20%**、任务价值 **10%**。

Both synthesized pools come from recursive task synthesis, which grows a curriculum rather than sampling tasks independently. Each round takes accepted tasks as seeds, picks a feasible rewrite operator, extends the reference solution with additional executable steps, then aligns the environment, verifier and public instruction to that longer workflow. Because difficulty is added to the executable path before the instruction is rewritten, horizons lengthen without the task degenerating into a longer prompt over the same behaviour.

Synthesis yield alone does not make a pool a usable RL signal, so T1-15k is the subset surviving an LLM audit. Instruction–verifier alignment carries the largest single weight: a mismatch between the public instruction and what the held-out verifier enforces is a hidden requirement, and it makes the task unusable because the agent is punished for failing a criterion it was never shown. The rest of the weight splits between verifier quality and solution quality, and tasks are hard-rejected for hidden requirements, test leakage, solution shortcuts, or verifiers too weak to validate the goal.

两个合成池都来自递归任务合成：它培育课程，而不是独立采样任务。每一轮把已被接受的任务作为种子，选择一个可行的改写算子，用额外的可执行步骤扩展参考解，再把环境、验证器与公开指令对齐到这条更长的工作流。因为难度是先加在可执行路径上、然后才重写指令，horizon 会变长，而任务不会退化成「同样行为的更长 prompt」。

合成产量本身不足以让一个池成为可用的 RL 信号，所以 T1-15k 是通过 LLM 审计后留下的子集。 指令与验证器的一致性是单项权重最高的维度：公开指令与留出验证器所强制内容之间的不匹配就是「隐藏需求」，它会让任务彻底不可用 ——因为 Agent 会因为一条它从未被展示的判据而受罚。其余权重分给验证器质量与解法质量；任务在四类问题上被硬性拒绝：隐藏需求、测试泄漏、解法投机，以及验证器弱到无法验证任务目标。

Selection: five stages over 15 rewrite rounds筛选：跨 15 轮改写的五个阶段

Aggregation with lineage; static pre-check with executable validation; a semantic pass that accepts, flags or rejects each task; an instruction-only repair that leaves the tests untouched; and a re-audit.

带血统信息的聚合；静态预检加可执行校验；对每个任务做通过、边界或拒绝判定的语义审查；只改指令、不动测试的修复；以及一次复审。

Stable MoE reinforcement learning

Between the trajectory a sandbox produces and the gradient the trainer applies lie dozens of turns, two independent execution stacks and a discrete router — each able to silently attribute the update to a policy that never generated the data. The agent therefore exchanges token identifiers rather than text, and receives back their log-probabilities and the expert routing chosen at every layer.

Three mechanisms build on that record: TITO for the token stream, R³ for expert routing, and a warm-started critic that turns the aligned trajectories into usable advantages.

在沙箱产出的轨迹与训练器施加的梯度之间，横着几十个轮次、两套独立的执行栈和一个离散路由器 ——它们每一个都可能悄悄把更新归因给一个从未产生该数据的策略。因此 Agent 交换的是 token id 而不是文本，并取回它们的对数概率与每层选中的专家路由。

三个机制建立在这份记录之上：TITO 管 token 流，R³ 管专家路由，以及一个 warm-start 的 critic，把已对齐的轨迹变成可用的优势。

The harness persists each assistant message as text and re-renders the whole history before every turn, and that round trip is not the identity. TITO removes it where we own both endpoints: the sampler returns token identifiers with their log-probabilities, encoding runs once per turn rather than once per history replay, and templates are pinned so rendering stays append-only.

Where the harness still forces a mismatch, the assembler repairs the boundary by the first case that applies:

strict — the turn is already an exact prefix of the next prompt.
normalized — trimming a bounded number of edge tokens restores the prefix, which keeps the repair auditable.
retokenized — the assembler matches in text space instead. The harness's re-tokenized copy never enters the training stream; the sampled identifiers do.
split — nothing holds, so a new chunk opens under the same trial.

An auditor then re-locates every completion inside the next prompt and reports zero drift inside the loss region.

harness 把每条 assistant 消息以文本持久化，并在每一轮之前重新渲染整段历史，而这个往返并不是恒等映射。在两端都由我们掌握的地方，TITO 直接消除它：采样器返回 token id 及其对数概率，编码每轮只做一次而不是每次历史重放都做一次，模板被钉住以保证渲染是 append-only 的。

在 harness 仍然强制产生不匹配的地方，装配器按第一个成立的情形修复该边界：

strict：该轮已经是下一轮 prompt 的精确前缀。
normalized：裁掉有界个数的边缘 token 即可恢复前缀，这让修复保持可审计。
retokenized：改为在文本空间匹配。harness 重新分词的那份副本永不进入训练流，进入的始终是被采样的 id。
split：以上都不成立，于是在同一试验下开一个新 chunk。

随后一个审计器在下一轮 prompt 内重新定位每一段补全，并报告损失区内零漂移。

TITO stitching for the retokenized case — **Figure 5 | TITO stitching, drawn for the retokenized case** — the hardest of the four. The re-tokenized copy never enters the training stream, whereas the sampled tokens do; observations and template glue enter masked, carrying context but no gradient.

**图 5｜TITO 拼接，按四种情形中最难的 retokenized 档绘制。** 重新分词的副本永不进入训练流，被采样的 token 才进入；观察与模板胶水以被 mask 的形式进入，只提供上下文而不提供梯度。

Router selection is a discrete top-k, so a numeric difference far below any tolerance one would place on a logit can swap a selected expert for its runner-up — substituting one sub-network for another between rollout and training.

R³ records the expert sets inference selected and constrains the training forward pass to reuse them: the selection comes from the record, while gating stays on the live scores of the current parameters. Replacing the top-k by a lookup removes the discontinuity, and reading gates from live scores keeps the router trainable. The masks are a by-product of a forward pass that already computed them, so capture costs transport but no arithmetic. Records ride the token pipeline's stitching and sharding, and a missing capture aborts the rollout rather than silently dropping the sample.

路由选择是一个离散的 top-k，因此一个远低于任何 logit 容差的数值差，就可能把某个被选专家换成它的次席——在 rollout 与训练之间把一个子网络替换成另一个。

R³ 记录推理选出的专家集，并约束训练前向复用它们：选择取自记录，而门控仍然落在当前参数的实时分数上。用查表替代 top-k 消除了那个不连续性，而从实时分数读门控又让路由器保持可训练。这些掩码是一次本来就已经算出它们的前向的副产物，因此记录只增加传输、不增加算术。记录接受与 token 流相同的拼接与分片；capture 缺失会中止 rollout，而不是静默丢样本。

Train-inference log-probability gap per step — **Figure 6 | Train–inference log-probability gap** over the production run, with an exponential moving average in bold.

**图 6｜生产实验中的训推对数概率差**，粗线为指数滑动平均。

Measured effect. Together the two mechanisms cut the mask-weighted mean gap by roughly a third. A residual is expected, since R³ aligns expert selection but not kernel numerics, and a slow upward drift is not a regression either: the gap also contains the distance the policy legitimately travelled during one update, and that term grows as the policy improves.

实测效果。两个机制合起来把掩码加权的平均 gap 压掉约三分之一。残差是预期的，因为 R³ 对齐专家选择但不对齐 kernel 数值；缓慢上漂同样不是退化：这个 gap 里还包含策略在一次更新中合法走过的距离，而随着策略变好，这一项会增长。

Configuration配置	mean gap平均 gap
T1 (TITO + R³)	0.013
without TITO and R³不用 TITO 与 R³	0.021

PPO here uses a separate, full-size critic that shares the actor's devices through forced offload. It trains first at each step and hands its pre-update values to the actor, so the advantage estimator and the value clip are anchored to one fixed function. Before any policy step, Critic Warm-Up trains the value network alone with the actor frozen; the dense-reward campaign then loads those weights only, so no optimizer moment crosses runs.

Explained variance decides whether this works, and it is unbounded below: a negative value means subtracting the critic adds variance to the advantage instead of removing it. Cold-started, it opens deeply negative and stays negative through much of the run; warm-started, it never goes negative. Reward nevertheless stayed flat across critic-tuning settings, which is what redirected our attention from the optimizer to the data and the reward.

这里的 PPO 使用一个独立的全尺寸 critic，通过强制 offload 与 actor 共享设备。它每步先训练，并把更新前的价值交给 actor，因此优势估计与价值裁剪都被锚定在同一个固定函数上。在任何策略步之前，Critic Warm-Up 先冻结 actor、只训练价值网络；稠密奖励实验随后只加载这些权重，因此没有优化器动量跨实验流动。

解释方差决定这套做法是否成立，而它下无界：负值意味着减去 critic 是在给优势增加方差而不是减少方差。冷启动时它一开始就深度为负，并在实验的很大一段里保持为负；warm-start 之后它从未转负。但奖励在各种 critic 调参设置之间仍然保持平坦，这正是把我们的注意力从优化器转向数据与奖励的原因。

Critic explained variance with and without warm start — **Figure 7 | Critic explained variance, with and without Critic Warm-Up.** Blue is the production dense-reward run; red is the cold-started critic. The dashed rule marks zero.

**图 7｜critic 解释方差，有与没有 Critic Warm-Up。** 蓝色是生产稠密奖励实验，红色是冷启动的 critic，虚线标出零位。

Dense verification reward design

Many tasks are too hard to solve outright, and under a binary reward those trajectories all score zero even when part of the requirements was satisfied — partial progress is invisible, and a batch costing hundreds of sandbox-hours returns one bit per trajectory.

We spend the verifier's full resolution instead: each trial's per-assertion outcome becomes a dense process reward, scored by the number of assertions satisfied on a scale fixed once for the whole run. Previously zero-reward trajectories then become graded supervision.

很多任务对模型来说太难、无法一次做完；在二值奖励下，这些轨迹全部得零，即使其中一部分要求已经被满足——部分进展完全不可见，而一个要花掉数百沙箱小时的 batch，每条轨迹只回报 1 bit。

我们改为花掉验证器的全部分辨率：每个试验的逐断言结果变成一个稠密过程奖励，按满足的断言个数、在一个为整场实验固定一次的尺度上计分。原本零奖励的轨迹于是变成了分级监督。

Absolute count, not ratio. Batches mix easy and hard tasks without a curriculum, and a ratio makes half of a long task look like half of a short one. Since harder tasks were synthesized with more checks, an absolute count better reflects verified progress.
Fixed global scale. The normalizer is chosen once from the pool's assertion-count distribution and never moves, since a per-batch maximum would let the reward scale drift and hand the critic an inconsistent target. A missing report falls back to the binary outcome, so a solved task never scores zero.
Credit assignment. Density is about value resolution, not temporal placement: the scalar still lands on the last response token and the critic distributes credit over the horizon. With one sample per task there is no group baseline, so the critic is the only baseline.

用绝对个数，不用比例。batch 里混合了易题与难题且没有课程安排，而比例会让「长任务完成一半」看起来等同于「短任务完成一半」。由于更难的任务在合成时就配了更多检查，绝对个数更能反映已验证的进展。
固定的全局尺度。归一化常数由数据池的断言数分布一次性确定，之后不再变动；因为按逐 batch 最大值归一化会让奖励尺度漂移，给 critic 一个不一致的目标。若逐断言报告缺失，该试验回退到二值结果，因此一个真正解决了的任务永不得零分。
credit 分配。「稠密」说的是取值分辨率而不是时间位置：标量仍然落在最后一个 response token 上，由 critic 把 credit 分配到整个 horizon。每个任务只有一条样本，因此没有组 baseline，critic 是唯一的 baseline。

Mean rollout reward over the production dense-reward run — **Figure 8 | Mean rollout reward over the production run**, before any normalization. Reward climbs over the first third of training and then holds a narrow band. The plateau is not stagnation: the held-out benchmark keeps improving through it.

**图 8｜生产实验中的 rollout 平均奖励**，在任何归一化之前。奖励在训练的前三分之一里上升，随后保持在一个很窄的带宽内。这个平台不是停滞：留出基准在整段平台期里持续改善。

Data-side filtering数据侧过滤

The primary defence against reward hacking is to reject gameable tasks before they enter the pool. The audit hard-rejects hidden requirements, test leakage, solution shortcuts, and verifiers too weak to validate the task's goal.

防止奖励作弊的首要防线，是在任务进入池子之前就拒掉可被钻空子的。审计对隐藏需求、测试泄漏、解法投机以及弱到无法验证任务目标的验证器一律硬性拒绝。

Reward-side monitoring奖励侧监控

A test-count reward can invite prolonging trajectories to brush more tests. We saw that at smaller scale, so we monitored turns and sequence length closely and observed no runaway growth (Figure 12b). Length shaping stays disabled.

test-count 奖励可能诱使 Agent 拉长轨迹去多蹭几个测试。我们在更小规模上见过这种行为，因此密切监控轮数与序列长度，没有观察到失控增长（图 12b）。长度整形保持关闭。

Shuffling is part of the reward design打乱是奖励设计的一部分

T1-15k is materialized in quality-rank order, so without shuffling every batch would come from one narrow quality band and the reward distribution would drift over the epoch. A seeded per-epoch permutation keeps those statistics stationary.

T1-15k 按质量排名顺序物化，若不打乱，每个 batch 都来自一个很窄的质量带，奖励分布会在一个 epoch 内漂移。逐 epoch 的带种子置换让这些统计量保持平稳。

03Results结果

Three held-out suites, none contributing tasks to any training pool: Terminal-Bench 2.1 (89 tasks, fraction resolved), Long-Horizon Terminal Bench (46 hard tasks paying continuous partial credit, reported as average reward), and Terminal-Bench Hard (100 tasks from an independently constructed, harder distribution).

三个留出套件，都没有向任何训练池贡献任务：Terminal-Bench 2.1（89 个任务，报告解决率）、 Long-Horizon Terminal Bench（46 个困难任务，给连续部分分，报告平均奖励）、以及 Terminal-Bench Hard（100 个任务，来自独立构造且更难的分布）。

All suites run one configuration: Terminus-2 hosted by Harbor, thinking disabled, proactive context compaction on, a fresh cloud sandbox per trial, and 3 attempts per task averaged. Two starting checkpoints appear: the base model and an SFT checkpoint denoted RST.

三个套件跑同一套配置：由 Harbor 承载的 Terminus-2，关闭显式 thinking，开启主动上下文压缩，每个试验一个新建的云沙箱，每任务 3 次尝试取平均。实验中出现两个起点 checkpoint：base 模型与记作 RST 的 SFT checkpoint。

Performance comparison across Terminal-Bench 2.1 and Long-Horizon Terminal Bench — **Figure 9 | Performance comparison.** Upper block: evaluated under the same harness as ours. Lower block: public leaderboard rows under other harnesses, shown for context only, since harness choice materially changes scores — Claude Opus 4.6 scores 70.1% under Claude Code against 63.8% under Terminus-2. A solidus denotes a size not recorded.

**图 9｜性能对比。**上半区在与我们相同的 harness 下评测；下半区是其他 harness 下的公开榜单行，仅作参照——因为 harness 选择会实质性地改变分数： Claude Opus 4.6 在 Claude Code 下为 70.1%，在 Terminus-2 下为 63.8%。斜杠表示规模未记录。

Terminal-Bench 2.1 standing of T1 against frontier models — **Figure 10 | Terminal-Bench 2.1 standing.** Blue bars trace our own pipeline: base 43.8%, the SFT checkpoint we initialize from 49.4%, and T1 after RL 64.0%. SFT contributes 5.6 points and RL a further **14.6 points**, so roughly three-quarters of the distance from base to final is earned by reinforcement learning on terminal tasks rather than by imitating demonstrations.

**图 10｜Terminal-Bench 2.1 上的定位。**蓝色条描出我们自己的流水线：底座 43.8%、我们初始化所用的 SFT checkpoint 49.4%、以及 RL 之后的 T1 64.0%。 SFT 贡献 5.6 个点，RL 再贡献 **14.6 个点**，也就是说从底座到最终的距离里大约四分之三是由终端任务上的强化学习挣到的，而不是由模仿示范。

Performance on Long-Horizon Terminal Bench — **Figure 11a | Long-Horizon Terminal Bench.** Base 18.9%, SFT 23.6%, T1 **27.9%**: RL adds 4.3 points and the full pipeline 9.0. T1 matches Gemini 3.1 Pro and exceeds GPT-5.4 (27.2%) and GLM-5.1 (26.7%).

**图 11a｜Long-Horizon Terminal Bench。**底座 18.9%、SFT 23.6%、T1 **27.9%**： RL 增加 4.3 个点，整条流水线增加 9.0 个点。T1 与 Gemini 3.1 Pro 持平，并超过 GPT-5.4（27.2%）与 GLM-5.1（26.7%）。

Terminal-Bench Hard resolved rate — **Figure 11b | Terminal-Bench Hard.** T1 resolves **38.0%** against 28.3% for SFT and 20.0% for base, exceeding DeepSeek V4 Pro at 36.0%.

**图 11b｜Terminal-Bench Hard。**T1 解决 **38.0%**，而 SFT 为 28.3%、底座为 20.0%，并超过 DeepSeek V4 Pro 的 36.0%。

Longer horizons and harder distributions therefore move in the same direction as Terminal-Bench 2.1. The comparison is not uniform across benchmarks, though: Claude Sonnet 4.6 scores below T1 on Terminal-Bench 2.1 yet records 37.3% on LHTB, so long-horizon performance warrants its own evaluation rather than being inferred from one ordering.

更长的 horizon 与更难的分布因此与 Terminal-Bench 2.1 同向移动。但跨基准的比较并不一致：Claude Sonnet 4.6 在 Terminal-Bench 2.1 上低于 T1，却在 LHTB 上记录 37.3%，所以长 horizon 的表现值得单独评测，而不能只从一个排序去推断。

Held-out Terminal-Bench 2.1 resolved rate over training — **Figure 12a | Held-out score over training.** The first evaluated checkpoint already clears SFT by 6.8 points; two late increases reach 61.8% and **64.0%**, both in the region where explained variance is highest.

**图 12a｜留出分数随训练步的变化。**第一个被评测的 checkpoint 就已超过 SFT 6.8 个点；两次后期上升分别达到 61.8% 与 **64.0%**，两者都落在解释方差最高的区域。

Average turns and total sequence length over training — **Figure 12b | Behavioral dynamics.** Turns and sequence length roughly double over training and then **flatten**, whereas the additive length penalty we tried produced unbounded growth.

**图 12b｜行为动态。**轮数与序列长度在训练中大致翻倍，随后趋平；而我们试过的加性长度惩罚则产生了无界增长。

Where the gain lands增益落在哪里

Terminal-Bench 2.1 by domain, turns and difficulty — **Figure 13a | Domain, interaction length, difficulty.** Average turns are 94.4 for T1 against 31.5 for SFT and 41.1 for base. All checkpoints reach 100% on Easy; Medium is 78/58/56% and Hard 33/30/20%.

**图 13a｜领域、交互长度与难度。**平均轮数 T1 为 94.4，SFT 为 31.5，底座为 41.1。三个 checkpoint 在 Easy 上都达到 100%；Medium 为 78/58/56%，Hard 为 33/30/20%。

Stage progression and per-domain comparison — **Figure 13b | Stages and domains.** RL accounts for **72.3%** of the total gain from base to final; against GPT-5.6 Sol, T1 reaches 100.0% on debugging and 88.9% on system administration.

**图 13b｜阶段与领域。**RL 占了从底座到最终总增益的 **72.3%**；对 GPT-5.6 Sol，T1 在调试上达到 100.0%，在系统管理上达到 88.9%。

Gains are uneven across domains. T1 also resolves every data-processing and machine-learning task, but GPT-5.6 Sol stays ahead on data science, scientific computing and mathematics, and file operations remain hard. The concentration of T1-15k in command-line engineering is one plausible explanation.
Most of the difficulty-level gain is on Medium. Easy leaves no measured headroom, Medium gains about 20 points over SFT, and Hard only about 3 — the hardest group remains the main source of failures.
Longer interaction is a cost as well as a capability. On representative unsolved tasks T1 spends hundreds of turns where a stronger baseline spends tens, and times out. Resource settings also differ across those comparisons, which limits attribution to capability alone.

领域增益并不均匀。T1 还解决了全部数据处理与机器学习任务，但 GPT-5.6 Sol 在数据科学、科学计算与数学上仍然领先，文件操作也仍然困难。T1-15k 集中在命令行工程是一个合理解释。
难度层面的增益主要在 Medium。Easy 已没有可测量的空间，Medium 相对 SFT 提升约 20 个点，而 Hard 只提升约 3 个点——最难的一组仍是失败的主要来源。
更长的交互既是能力也是代价。在代表性的未解决任务上，T1 花掉数百轮，而更强的基线只用几十轮，并最终超时。这些比较中的资源设置也不同，这限制了把失败完全归因于能力。

04Cite this work引用

If T1 is useful in your research, please cite the technical report.

如果 T1 对你的研究有帮助，请引用这份技术报告。

BibTeX

@misc{yang2026t1terminalagentreinforcement,
      title={T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks}, 
      author={Junyao Yang and Yucheng Shi and Zhongzhi Li and Ruhan Wang and Zongxia Li and Haitao Mi and Leowei Liang},
      year={2026},
      eprint={2609.11042},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2609.11042}, 
}

Also available on arXiv as arXiv:2609.11042 [cs.LG].

也可在 arXiv 上获取：arXiv:2609.11042［cs.LG］。