Axion 1.0: Open Agent Foundation Models, Simplified
ByteDance's Axion 1.0 demonstrates that a three-stage training pipeline—two lightweight SFT passes followed by RL—can produce agent foundation models competitive with frontier commercial systems, while releasing the complete reproducible recipe including data construction, verification, and training code.
Transcript
Jessica Okay, so ByteDance just dropped Axion 1.0. Three training stages. That's it. Two SFT passes, then RL. No mid-training, no DPO, no on-policy distillation. And it's competitive with frontier commercial models on agent benchmarks.
Cathy Right. The paper's central question is basically: how much of the standard elaborate pipeline is actually necessary? Their answer: way less than people think, if you obsess over data construction instead.
Jessica Eighteen thousand SFT examples total. Across both stages. That's… tiny. Most teams are throwing millions of examples at this.
Cathy Mm-hm.
Jessica And they're not just releasing weights. The whole recipe—data generation, filtering, verification, prompts, training code. GitHub repo is bytedance/axion1_0. Project page at axion1-0.github.io with detailed benchmark breakdowns.
Cathy That's the part that matters for reproducibility. Agent capabilities depend as much on the interaction design and training data as architecture. Releasing weights alone never told the full story.
Jessica Okay, so walk me through the two SFT stages. First one is self-contained reasoning—model-aware data construction. What does that actually mean?
Cathy They synthesize challenging problems, then adversarially strengthen them based on the base model's own successful solution strategies. So the data construction adapts to what the model already does well—and where it fails.
Jessica Wait, they use the base model to generate the training data that trains the base model? That's… circular in a clever way.
Cathy It's model-aware curriculum generation. The base model's strengths and weaknesses shape the synthetic problems. You're not just dumping generic reasoning data—you're targeting the specific capability gaps this model has.
Jessica And that's before any tool exposure. Pure reasoning first. Then stage two extends to tool-dependent reasoning.
Cathy Right. Scalable pipeline for diverse, automatically verifiable tasks. Grounded in real-world data sources and deterministic computation. That's the key—verifiable outcomes without human annotation bottlenecks.
Jessica So they've basically cracked the verification problem for tool use by restricting to tasks where the answer is deterministically checkable. Code execution, API calls with known outputs, that kind of thing.
Cathy Exactly. And the progression makes sense: independent reasoning → grounded interaction → RL consolidates both over longer trajectories. The RL stage is what stretches the horizon.
Jessica Figure 1 shows Axion 1.0-122B and 35B against leading open and proprietary models across six benchmarks—general reasoning, scientific reasoning, search, agentic problem solving. Blue and teal bars are competitive or superior across the board.
Cathy The baseline comparison notes matter though. Evaluated by the Axion team, other scores from official reports. Settings may differ. They're transparent about it on the project page—HLE full benchmark vs HLE-130 subset, BMB human-difficult subset, best of three runs or five for BMB.
Jessica Good. They're not hiding the evaluation caveats. That's… refreshingly honest for a release this strong.
Cathy The training setup is reproducible too. 122B SFT recipe uses 32 GPUs, 128K context window, global batch of 8. Swift framework. The repo has the training guide for environment setup, two-stage SFT, checkpoint export.
Jessica So a team with 32 GPUs can reproduce the 122B training. That's… not trivial, but it's not 'only frontier labs' territory either. The 35B version would be more accessible.
Cathy And the simplicity of the recipe means it transfers across base models. That's the whole thesis—you're not locked into a fragmented sequence of specialized stages that only work for one architecture.
Jessica This feels like the 'boring but works' signal we talked about episode 1001. When the argument shifts from vibes to which task class you'd route to which model, the product is doing its job.
Cathy Stop it, you're making it sound almost boringly sensible. Again.
Jessica I know. It's deeply rude of them. We can't even get a nice chaotic launch to argue about.
Cathy Honestly, that's probably the strongest signal here. The recipe simplicity is the feature, not a bug.
Jessica Okay, Build Next: if you want to reproduce or extend this, the repo is bytedance/axion1_0 on GitHub. Project page at axion1-0.github.io has the full benchmark breakdown with caveats. Training guide in the repo walks through environment setup, two-stage SFT, checkpoint export. Uses Swift.
Cathy And the paper's on arXiv: 2611.03233. Data construction pipelines for both stages are the real takeaway—model-aware synthesis for reasoning, verifiable tool tasks for interaction. That's what ports to other bases.
Jessica Three stages. Eighteen thousand examples. Competitive with frontier. Full recipe open. …Yeah, I don't know what to argue about here.
Cathy Neither do I. That's the point.
Jessica Alright. Catch you on the next one, Cathy.
Cathy Yeah. See you.