Zero-Training TabPFN and TabICL Beat Tuned XGBoost on All 14 Benchmark Datasets

Two newer tabular models, TabPFN and TabICL, were benchmarked against a tuned XGBoost across 14 datasets from the Grinsztajn benchmark under identical train-test splits and time constraints. Unlike XGBoost, both models make predictions without any dataset-specific training, relying instead on prior-fitted networks. The zero-training models outperformed tuned XGBoost on all 14 out of 14 datasets tested. However, the comparison also highlighted trade-offs in latency and GPU memory usage associated with the newer approaches. The author notes that gradient boosting methods like XGBoost may still be preferable in certain real-world scenarios despite the benchmark results.