Developer builds fully local AI assistant with Ollama, LangChain, and voice support

A developer has built a fully local personal AI assistant from scratch using Ollama for the language model, LangChain/LangGraph for agent logic, faster-whisper for speech transcription, and Piper for text-to-speech synthesis, with all components running on-device. The project revealed significant limitations of small LLMs (around 8B parameters) in tool calling, where models sometimes printed raw JSON function calls as plain text instead of executing them properly. Smaller models also proved 'lazy' when generating structured outputs, such as writing placeholder comments instead of actual code when routed through tool calls — a problem that was solved by bypassing structured tool calling and using regex-based parsing instead. The developer also found that complex multi-step tasks like 'research and save' worked better when web searches were executed deterministically before invoking the LLM, rather than relying on the model to chain tools autonomously. Memory management emerged as another key challenge, requiring three distinct layers — in-session context, persistent conversation checkpoints via SQLite, and a separate long-term facts store — along with a summarization middleware to prevent unbounded context growth.