Open-Source RAG Chatbot Starter Uses FastAPI and ChromaDB With No API Key

A developer who builds production RAG and LLM systems has released a free, open-source starter template for creating document-based chatbots using FastAPI and ChromaDB. The system uses Retrieval-Augmented Generation, a technique that retrieves relevant document chunks before passing them to a language model, reducing hallucinations and enabling source citations. The stack includes SentenceTransformers for embeddings and supports any LLM backend, including locally run models via Ollama, meaning no API key or internet connection is required. Key design considerations highlighted include chunk size tuning, overlapping chunks, OCR handling for scanned PDFs, and cross-encoder re-ranking to improve retrieval quality. The project is licensed under MIT and can be cloned and run locally with a single Docker command.