② Task filtering: each rollout group has a signal. Only tasks with initial pass rates between 15%–85% (measured with at least four samples under the same conditions as training) are retained, reducing the proportion of zero-signal rollout groups from 48% to 10.2%.
③ Reward: only give points for correct answers. Each task must pass through nine validation stages before being added to the pool, and use calibrated partial scores for the final part; efficiency rewards for shorter answers reduce the number of tokens used per solved task to 0.84 times the original, without decreasing accuracy.
④ Harness: long-term tasks fit into the context, and training and evaluation are consistent. Context compression, output redirection to files, and background execution reduce the cost of a single experiment by about 33%; training and evaluation share the same harness, allowing GLM-5.3-Flash to be 4.4 points higher than Claude Code even before any RL.
⑤ RL objective: outdated FP8 samples can still drive learning without drifting. GRPO combines truncated importance sampling, top-k score centering, success offset, and loss lower bound, maintaining stability without KL penalty and entropy reward, with only 1%–3% of tokens being truncated.
Infrastructure: fully asynchronous RL
Core experience 1: idle GPUs cost more than outdated rollouts—let sampling and training run at full speed.
Training and sampling are split across different GPUs. Half of the GPUs use the slime framework to train in BF16, while the other half run the SGLang inference engine, with weights and KV cache in FP8. After each training step, new weights are synchronized to the rollout engine via NCCL, and training and sampling do not block each other (Figure 2).
Using outdated rollouts for training. Since sampling is continuous, a round (turn) may be generated by weights that are several steps behind the training; only when it comes from weights older than six versions will it be discarded. The offset caused by outdated rollouts is minimal: the log probability difference between the trainer and the rollout is 0.065 when outdated by one version, and only rises to 0.08 when outdated by four to six versions.
Fast and isolated rollouts. Utilizing the model's native multi-token prediction (MTP) head for speculative decoding, boosting decoding speed by 1.38x. Each rollout and scoring is executed in an independent Firecracker microVM via AgentENV, with attempts isolated from each other, reducing the percentage of rollout groups lost due to infrastructure errors from 12% to 0.5%.
Figure 2: Fully asynchronous RL. Rollout and training each take up half of the GPUs, not waiting for each other. The inference engine continuously samples, and each attempt runs in an independent microVM derived (forked, taking less than 100 milliseconds) from its task's parent VM; completed groups queue up to wait for the trainer to take a batch. All groups with the same score are discarded, and rounds sampled with weights older than six versions do not produce gradients. After each update, the trainer pushes new weights to the inference engine.
Task filtering: only train where the model is uncertain.
Core experience 2: only spend rollouts on tasks where the model is uncertain, and measure this uncertainty under the same conditions as training.
GRPO learns by relying on the relative differences between multiple attempts of the same task. A task that a model can always solve or never solve will consume an entire rollout but provide zero learning signal . Therefore, the team conducts at least four rollouts for each candidate task to screen it out, retaining only those tasks with an initial pass rate between 15% and 85%. The screening strictly follows the training configuration —the same harness, inference strength, context window, rollout budget, and sandbox—because the pass rate of a task depends on each of these. This filtering reduces the proportion of zero signal rollout groups from 48% to 10.2%. In the final training data, long-horizon agent tasks account for approximately 52%, and software engineering tasks account for approximately 48% (Figure 3).
Figure 3: Composition of training data. Orange represents long-horizon agent tasks (totaling 52%): long-horizon agent tasks built in-house (30%) and their difficult variants (11%), long-horizon programming skills (9%), and public MiMo-TB and Tasksmith (each 1%); Blue represents software engineering tasks (totaling 48%): public MiMo-code (18%), and in-house SWE tasks (18% primarily in Python, 12% covering TypeScript, Go, Python, JavaScript, and Rust).
Scoring and Reward: First validate the scoring system, then discuss preventing reward speculation
Core Experience 3: Tasks that models never solve are often broken tasks—validate each scoring system before the reward is given to the model.
When the team reviewed tasks that were never solved, they found that nearly half were "broken" rather than "difficult" : the task itself had flaws, relied on fragile tests that could misjudge correct solutions, or contained "protected file" traps. Some scoring systems would give partial credit just for matching the output format, or would crash upon encountering real agent outputs. To prevent these issues, each new task must pass through nine validation stages (Figure 4) to confirm that the task is solvable according to the problem description, the sandbox is closed and stable, and the scores given by the scoring system are accurate.
Figure 4: Nine stages that new tasks must pass through before entering the training pool. Blue represents tasks (2 stages): the reference solution can pass in the training sandbox, and the task goes through exactly the same training process; every requirement checked by the scoring system is explicitly stated in the problem description. Green represents sandbox (3 stages): runs offline; starts reliably every time without freezing; the environment does not leak answers. Orange represents scoring system (4 stages): attempts that do nothing or only submit sample code receive 0 points; tests are not fragile, and any correct solution can pass; even the most malformed real outputs can be scored without crashing; the same output always receives the same score.
Reward Design. Each attempt is scored once at the end, and the final reward is in the range [0, 1]. A fully solved task receives 1.0; incomplete attempts are given partial scores based on the number of passed validation checks, and software engineering tasks are scored based on the number of fixed failed tests. Solved attempts can also receive a maximum of 0.2 length reward , calculated based on the number of tokens saved compared to the longest solution in the same group. This shaping reduces the number of tokens used per solved task to 0.84 times the original, without decreasing evaluation accuracy.
Harness: Built for long-horizon attempts, training and evaluation are completely consistent
Core Experience 4: The Harness is part of the model—use the same one for training and evaluation.
The team's harness follows the same conventions as Claude Code: four tools (Bash, Read, Write, Edit), native tool calls, preservation of the reasoning history, and context compression. The key is that training rollouts and evaluation trials are driven by exactly the same harness , and automated unit tests ensure consistency between the two in terms of code and prompt. Before any RL, this harness allows GLM-5.3-Flash to outperform its official Claude Code results by 4.4 points on Terminal-Bench 4.0 () (Table 2).
Table 2: Performance of GLM-5.3-Flash using Claude Code (official leaderboard results) and the team's harness on Terminal-Bench 4.0. Standard time limits, with cost per single trial and cost per solved trial calculated based on public prices.
Long-horizon attempts must also be affordable. Context compression is triggered when the occupied context reaches 85% of the context window, preserving the prefix cache by appending a section summary prompt to the end of the existing conversation history; commands write standard output to a file and only return a truncated preview with the filename. Long-running commands are moved to the background for asynchronous execution rather than being terminated upon timeout. These features collectively reduce the cost per trial by about one-third . Additionally, the prompt specification for efficient workflows (in-place editing, write once, no sleep polling) saves another 16%.
RL objective: Learning from outdated samples without drifting
Core experience 5: Correct outdated, low-precision samples instead of discarding them, and treat rising entropy as an early warning.
Rollouts are sampled at FP8 precision, using weights at most six versions behind, slightly differing from the currently trained BF16 policy. The team does not discard these outdated trajectories, but rather directly corrects them in the training objective . The core algorithm is GRPO, with 12 attempts per task and one optimizer update per step; the advantage value for each attempt includes both a reward term and a length reward term:
A = (R′ − mean R′) / std R′ + (b − mean b) / 0.5
(mean and standard deviation are calculated over the 12 attempts for the same task)
R′ = 1.5 (when the attempt solves the task); otherwise R′ = 0.5 × R (R is the final reward in [0, 1] from above)
b = 0.2 × (1 − tokens / maximum token number in the group for solved attempts), calculated for solved attempts within the group if there are at least two solved attempts, otherwise b = 0.
Truncated Importance Sampling (TIS). Each token is weighted by the ratio of BF16 trainer probability to FP8 rollout probability, with the ratio truncated to [0.5, 2.0] to limit the impact of individual tokens; only 1%–3% of tokens reach the truncation boundary.
Score Centering (SC). The rollout engine returns the top-32 log probabilities for each token, and the trainer subtracts the expected score of the sampler on these candidates from the policy gradient score of each token. This eliminates the drift toward the outdated FP8 sampler that would still remain even with truncation, at the cost of a 7.6% increase in per-step time.
Success Offset. Fully solved attempts (R = 1.0) are counted as 1.5 in the advantage, while unsolved attempts are counted as half of their partial score: the best "almost" (R = 0.5) is only counted as 0.25, while a fully solved attempt is counted as 1.5. Partial scores can shape learning, but are not equated with true solving.
Loss Normalization Lower Bound. The loss for each attempt is normalized by its token count, with the denominator being at least 2,048 tokens, preventing degenerate rollouts with only a few tokens from gaining excessive single-token weights and dominating the gradient.
With these corrections, training does not use KL penalty nor entropy reward . In healthy training, the policy entropy gradually decreases from approximately 0.46 to 0.33; an increase in entropy is an early warning signal of training instability. Experiment: three benchmarks are comprehensively improved, standing on the cost-performance frontier. FireFlash is evaluated using the same harness as during training and compared with its starting point GLM-5.3-Flash's official results and other models' public results (Figure 5). FireFlash exceeds the baseline on all three benchmarks: Terminal-Bench 4.0 improves by 10.4 points, surpassing Fable 5 and the full-sized peer model GLM-5.3; DeepSWE v1.1 improves by 5.4 points, surpassing Fable 5.1; SWE-Bench Pro Verified improves by 5.1 points, surpassing Kimi K3 and GPT-5.6 Sol. Figure 5: Pass@1 on three benchmarks. Orange represents FireFlash, blue is its starting point GLM-5.3-Flash's official results, and the rest are public results of other models. Cost and performance: On Terminal-Bench 4.0, FireFlash is located on the cost-performance Pareto frontier (Figure 1): among over forty models and inference intensity settings in the figure, only four have a lower cost per trial than it, and their scores do not exceed 35.8%. Based on GLM-5.3-Flash's public price, FireFlash's cost per trial is $0.72, while Fable 5 at the highest inference intensity is $22.02; the next point on the frontier, GPT-6.1 Terra, scores 55.6% with a cost of $1.49. All models with scores higher than FireFlash have at least double the cost per trial; the full-sized peer model GLM-5.3 costs $8.27 per trial at the highest inference intensity, yet scores only 41.8%. Conclusion: FireFlash demonstrates that targeted post-training can transform open-source weight models into highly capable domain-specific agents, achieving frontier-level performance with a small portion of inference cost. The authors believe that joint optimization of models, training environments, harnesses, and inference systems is another path to building powerful and efficient agents beyond mere scaling. FireFlash is the first step in this direction, and the team will next extend this approach from programming and terminal tasks to more complex, longer-term, and requiring repeated execution tasks. Author introduction: Wenbo Hu, a Ph.D. student in computer science at UCLA, advised by Professors Kai-Wei Chang and Nanyun Peng. His research focuses on agentic reinforcement learning, with interests in large language models, multimodal systems, and agents. His representative works include Tev, MQT-LLaVA, 3DLLM-Mem, etc. Other authors of this paper are Jesse Cai, Po-nien Kung, Yuxuan Lu, Di Wu, Huimin Zeng (sorted alphabetically by surname), and Professors Nanyun Peng, Dakuo Wang, and Kai-Wei Chang.
