StepFun Releases Step 5 Preview MoE Model With 1M Context

StepFun has released Step 5 Preview, its new flagship model designed for agentic workloads in software engineering, professional knowledge work, and finance. The primary selling point of Step 5 Preview is its cost-effectiveness, with the StepFun team asserting that the model delivers comparable intelligence at a substantially lower task cost, framing this as achieving a 'Pareto frontier'. The model is currently available as a hosted API and on the StepFun platform, with open weights slated for release on October 15, 2026. The architecture of Step 5 Preview is a sparse Mixture-of-Experts (MoE) model, boasting approximately 600 billion total parameters while activating around 27 billion parameters per token, representing roughly 4.5% of the total weights utilized for each token. This configuration is designed to enhance efficiency for specific tasks. The official model documentation specifies a context window of 1 million tokens, supporting input modalities of text, images, and video, with text as output. It is engineered for reasoning across low, medium, and high complexity levels and includes features such as streaming, tool calling, JSON Mode, JSON Schema, and prompt caching. In performance demonstrations on research tasks, the StepFun team reported that the model successfully coordinated 950 web fetches within a single agent action, showcasing its capability for complex, multi-step operations. Furthermore, the StepFun team documented an integration with Claude Code through its Step Plan, indicating interoperability with other AI development tools. The model's architecture is characterized as 'Narrow and Deep,' with 92 Transformer layers stacked in a narrow-deep layout, a design choice that the research team posits facilitates longer paths for implicit multi-hop reasoning, which is particularly beneficial during long prefill phases where agents engage in searching, code execution, and processing tool returns. The training methodology for Step 5 Preview leans on on-policy, long-horizon reinforcement learning, with StepFun citing bit-wise train and inference alignment across MoE routing as a key technique. Additional listed techniques include MTP-3 speculative decoding, FP8 MoE, and KV-cache offload. StepFun reports that these optimizations result in more than a 3x end-to-end speedup for long-horizon reinforcement learning tasks. The significant context window of 1 million tokens is a notable advancement, enabling agents to process and retain information over much longer interactions or extensive datasets, which is crucial for complex problem-solving and maintaining state in extended agentic workflows. The model's ability to handle diverse input types and its focus on cost-efficient, high-intelligence output position it as a competitive offering in the AI agent space.