Kimi K3.1 Pro and Claude Fable 5.2 Tested on 19 Tasks: Capability Gap Minimal

A developer ran 19 identical tasks through Kimi K3.1 Pro and Claude Fable 5.2 on the same day via one OpenAI-compatible endpoint, changing only the model name. Both models passed nearly all tasks, scoring 17/19 and 18/19 respectively, with the misses attributed to slips rather than capability limits. Kimi generated about 84% of its output as hidden thinking tokens, which drives its cost, while Fable produced no thinking tokens. On a per-visible-token basis, Kimi cost roughly $63 per million tokens versus $40 for Fable, reversing the apparent price advantage. Both models handled roughly 14K-token long-context inputs correctly, but the test did not examine million-token capacity, and the author noted that temperature parameters are inconsistently supported across providers.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in