VIDRAFT wins Gemma-4 inference challenge with 510.58 TPS on single A10G GPU
VIDRAFT claimed first place in 'The First Gemma Challenge', a constrained AI inference competition requiring participants to optimize speed solely through software on a single NVIDIA A10G GPU running Google's Gemma-4-E4B-it model. The winning submission achieved 510.58 tokens per second while maintaining a perplexity score of 2.3930, comfortably within the quality threshold of approximately 2.42. A rival entry recorded a faster raw speed of 535.91 TPS but was disqualified after its perplexity exceeded the limit, making VIDRAFT's result the verified state-of-the-art. Key optimizations included a tuned sliding attention window of 188 tokens, an empirically selected centroid top-k value of 49, and a warm-up routine that absorbed GPU compilation costs before timing began, worth roughly 15 TPS. Speculative decoding with seven draft tokens per pass was also enabled, collectively allowing VIDRAFT to pass a blind re-evaluation against a held-out prompt set the team had never seen.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in