Mistral's Timestral Forecaster Ranks First by Narrow Margins, Test Shows Context Length Matters

Mistral released Timestral, its first family of open-weight time-series foundation models, on October 15 in four sizes ranging from 5M to 209M parameters. The company says the models rank first across major public forecasting benchmarks, a claim the model card supports. However, the margins over the runner-up are very small, such as 0.003 on the headline MASE metric on GIFT-Eval. A developer testing the 209M model on two weeks of Paris temperature data found that a model given one summer of history lost to a simple repeat-yesterday baseline. With three years of history, the same model outperformed the baselines by about a third. The developer also noted that the training data was mostly synthetic, which the author suggests affects how the benchmark results should be interpreted.