StepFun Releases Open Weights for Step 5 Preview, a 600B-Parameter MoE Model

StepFun has published the open weights of Step 5 Preview, its flagship foundation model aimed at long-running agentic orchestration and multi-hop analytical tasks. The model uses a sparse mixture-of-experts design with 600 billion total parameters, activating about 27 billion per token, and supports a context window of up to 1 million tokens. Its architecture alternates sliding-window and global attention layers across 92 layers, which keeps memory use manageable for long contexts. Released in BF16, FP8 and NVFP4 formats on Hugging Face and ModelScope, the weights let companies run the model on their own infrastructure instead of relying on closed APIs. The developers say the NVFP4 checkpoint fits on four Hopper or Blackwell accelerators at roughly 340 GB of VRAM and delivers over 110 tokens per second.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in