Reading the following blog post:

Overview. SWE-1.7 pushes the cost-performance Pareto curve for agentic software engineering by running extensive RL on top of Kimi K2.7 (itself already heavily RL post-trained). The gains challenge the “post-training ceiling” — consistent with the predictable scaling of RL — showing RL scales capability well beyond prior expectations.

Numbers: 42.3% on FrontierCode 1.1 Main, above base Kimi K2.7 (30.1%) and Claude Opus 4.7 (38.5%), just behind GPT-5.5 (43.0%) and Claude Opus 4.8 (46.5%). Also 81.5% on Terminal-Bench 2.1 and 77.8% on SWE-Bench Multilingual.

FrontierCode 1.1 Main benchmark comparison

1. Stabilizing Training and Preserving Entropy

Long asynchronous RL runs hit two walls: entropy collapse (model stops exploring, reward plateaus) and instability from KL mismatch between the trainer policy and the inference sampling policy. Importance sampling and quantization-aware training handled this at small scale; larger scale needed deeper algorithmic fixes. (The first-order-approximation view of stable RL frames this trainer–sampler mismatch precisely.)

The math of entropy collapse. Very low-probability tokens often lead to off-track, low-reward trajectories. Softmax then sharpens the distribution when penalizing them, killing entropy.

Take three tokens with logits $x_1 > x_2 \gg x_3$ and $p_i = \frac{e^{x_i}}{e^{x_1} + e^{x_2} + e^{x_3}}$. If the model samples the low-probability token 3, the gradient of its log-prob w.r.t. logits is:

\[\nabla \log p_3 = \begin{bmatrix} -p_1 \\ -p_2 \\ p_1+p_2 \end{bmatrix}\]

The update follows $\Delta x_i \propto \hat{A}\, \nabla \log p_3$. Since this is a low-reward trajectory, $\hat{A} < 0$:

\[\Delta x_1 \propto |\hat{A}| p_1, \quad \Delta x_2 \propto |\hat{A}| p_2, \quad \Delta x_3 \propto -|\hat{A}|(p_1+p_2)\]

Because $p_1 > p_2$, $x_1$ grows faster than $x_2$ while $x_3$ is heavily penalized. The dominant token’s lead widens, the distribution sharpens, exploration dies.

Solution — top-$p$ sampling + replay. Enforce top-$p$ sampling during rollouts so low-prob tokens never become optimization targets. To fix the resulting train-inference mismatch, use sampling distribution replay: record a “kept-set” of available tokens during rollout and renormalize probabilities with those masks in the trainer (a token-level cousin of rollout routing replay for MoE RL).

  • Bonus: tokens above the top-$p$ threshold have a keepset of 1 → renormalized probability of 1 → zeroed gradient. High-probability tokens drop out of gradient computation, cutting gradient noise and focusing the optimizer on high-signal tokens.
  • Further stabilized by eliminating non-deterministic trainer ops (numeric determinism matters for RL parity) and using the Muon optimizer.

2. Globally Distributed RL Infrastructure

Trillion-parameter RL is usually bottlenecked by the need for a massive single-fabric cluster. Cognition decoupled it into one centralized trainer cluster and widely distributed inference clusters across three continents — the async trainer/generator decoupling explored in StreamRL and RL Systems Mind the Gap, taken cross-continental.

Decoupled globally distributed RL infrastructure

  • Weight streaming: every $K$ gradient steps the trainer computes and transmits a compressed weight delta, shrinking transfers by over 99%.
  • In-place updates: deltas are distributed via cloud object storage. Inference engines prefetch into CPU memory while still serving, then pause just 3–4 seconds to apply weights in place. For a 1T model, cross-continental updates finish in 1–2 minutes end-to-end.
  • Fault tolerance: inference failures are cheap — NVIDIA Dynamo manages replicas and reroutes dead sessions. Trainer node failures (which could stall everything) are mitigated by asynchronous local-disk checkpointing every step plus peer shard replication, rebuilding state in seconds.

3. Long-Horizon Self-Compaction

SWE-1.7 targets complex tasks (up to six hours during training), so it inevitably exceeds the raw context window.

  • Self-compaction: near the context limit, the model summarizes its working state and resumes from its own summary. RL optimizes it to write succinct summaries and use them well.
  • Alternating length penalty: reasoning models tend to bloat length under RL. Cognition alternates: unconstrained phases chase task success, while budget phases penalize a weighted cost of tokens, turns, and total tool-call time. This compresses easy tasks while preserving long-horizon persistence on hard ones.

4. Data Quality and Behavioral Impacts

The pipeline prioritizes high-learning-signal tasks (low solve rates) and rigorous automated verifiers that minimize false positives/negatives. Anti-cheating measures — network-restricted sandboxes, isolated grading paths, immediate 0-reward for cheating — keep the model aligned.

Behavioral insights:

  1. Condensed chain-of-thought: the budget phases yield an efficient CoT — far fewer function words, roughly half the words per sentence vs. the base model.
  2. Extensive exploration: SWE-1.7 investigates codebases exhaustively before acting — probing edge cases, hypothesizing root causes, and writing small Python scripts to settle ambiguities rather than guessing.
  3. The “scope” tradeoff: deeper reasoning about beyond-the-ask requirements means it modifies more files and writes more tests than strictly needed. Cognition frames this as an industry-wide trend — more reasoning correlates with expanded modification scope, an open axis for future optimization.