Developer shares key lessons from 22-day hands-on AI systems building sprint

A software developer documented a 22-day self-directed learning sprint focused on building practical AI systems, covering topics from LLM fundamentals to retrieval-augmented generation (RAG) and automated evaluation. The experiment corrected several common misconceptions, including the true purpose of RAG — retrieving relevant facts at request time rather than serving as a privacy tool — and clarified how embeddings differ from next-token prediction mechanisms. Using tools like Ollama and local models, the developer demonstrated that LLMs are stateless by design, with apparent memory in chat apps being the result of deliberate state management by the application layer. Tokenization tests revealed that prompt costs and latency vary significantly by language, with Hindi requiring roughly 2.5 times more tokens per word than English. The overall takeaway was that a useful AI feature requires the same engineering discipline as any production software: defined inputs, constrained behavior, observability, and measurable evaluation criteria.