Lessons From Building a Small RAG Chatbot That Answers From Your Own Documents

A developer built a Retrieval-Augmented Generation system during an internship at Valentius Kryptix to test whether a language model improves when it reads user documents first. The system retrieves relevant passages from two documents of about 9,000 words, split into roughly 2,000-character overlapping chunks stored in a local Chroma database, and passes them to Gemini with the question. In a single test about a prompt engineering report, the plain model said it had no access to the file, while the RAG version retrieved the correct passage and gave the right answer. The author highlights three lessons: chunking involves trade-offs between noise and lost context, retrieval quality should be checked before blaming the model, and clear instructions to answer only from context improve trust. The author notes the project is small and tested only once per question, with plans to explore re-ranking and chunk sizes next.