Tiny 1.5B model matches a 7B on text-to-SQL tasks using smart training and voting

A developer fine-tuned a small 1.5-billion-parameter language model (Qwen2.5-Coder-1.5B) for text-to-SQL tasks using continued pretraining, supervised fine-tuning, and reinforcement learning with LoRA adapters on a single GPU. An early 0.5B prototype scored just 6.4% accuracy — worse than the untrained base model — because evaluations were mistakenly run on training data rather than a held-out set. Once a proper evaluation harness was built to compare query outputs against real databases, accuracy climbed from 6.4% to 68.1% on the upgraded 1.5B model. A majority-voting technique — generating eight answers per question and selecting the most agreed-upon result — pushed the score to 71.5%, edging past the 7B model's 71.2% single-attempt score. The project illustrates a growing trend of targeting small, task-specific models with robust surrounding systems to outperform much larger general-purpose ones.