How to Pick a Base LLM for Fine-Tuning: Context, Language, Cost and Licensing

Choosing the base model is presented as the most consequential early decision in fine-tuning a large language model, more so than adapter methods or dataset preparation. The article advises starting with a single measurable business objective that guides model selection and evaluation. It compares model architectures, noting that decoder-only models suit text generation, encoder-decoder models suit summarization and translation, and multimodal models add image reasoning. Context window length, language coverage, and GPU hardware requirements are highlighted as key selection factors, with Llama 4 Scout cited for very long documents and Qwen 3 and Aya for multilingual work. The piece also warns that licence terms, such as Llama's commercial restrictions compared with more permissive options, should be checked before deployment.