How Local AI Models Actually Work: A Developer's Deep Dive with Ollama

A developer running experiments on an NVIDIA DGX Spark machine spent Week 2 of a 32-week project exploring what happens beneath the surface of a local AI model API. Using Ollama as the runtime, they demonstrated the distinction between a model — which is static weight data stored on disk — and the runtime, which loads those weights into memory and generates text from prompts. By querying Ollama's local API, the developer retrieved key technical details about the Phi-4 model, including its 14.7 billion parameters, 16,384-token context length, and Q4_K_M quantization format. They also inspected the model's manifest file, revealing that a packaged model consists of separate blobs for the weights, prompt template, license, and default parameters. The exercise aimed to build foundational understanding of how AI model packaging and runtimes function before advancing to more complex GPU-level topics.