Empero AI Releases Qwythos-9B-v2, Eliminating Looping Bug in 1M-Token Model
Empero AI has launched Qwythos-9B-v2, an updated version of its Qwythos-9B large language model designed primarily to fix repetitive looping behavior seen in the previous release. The looping issue, which affected 6.7% of greedy-decoding generations, has been reduced to zero in the new version using a targeted technique called Final-Token Preference Optimization (FTPO). The fix involved fine-tuning the model on roughly 2,000 automatically mined preference tuples using LoRA, without degrading its core reasoning capabilities. The update also restores a previously missing multi-token-prediction module and refines identity management so the model no longer introduces itself unprompted. Internal benchmarks show that most performance metrics were maintained or improved, though HumanEval code scores dipped slightly to 77.4%.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in