Maple Claims Gemma 4 26B Runs at 45 Tokens/Sec on a 6GB GPU via MoE Pruning
A Hacker News post introduces Maple, a project that applies sub-2-bits-per-weight mixture-of-experts pruning. The developer says the technique lets Google's Gemma 4 26B model run at roughly 45 tokens per second on a GPU with 6GB of memory. The project is shared as a GitHub repository, and the post received 3 points and one comment at the time of listing. The source material is limited to the repository link and discussion metadata, so the performance claims have not been independently verified.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in