Engineer Builds Hybrid Agentic RAG System Combining FAISS and BM25 for Smarter LLM Retrieval

A developer has designed and documented an Agentic Hybrid Retrieval-Augmented Generation (RAG) architecture that merges dense semantic search via FAISS with exact keyword matching using BM25, aiming to overcome limitations of traditional static RAG pipelines. The system uses an autonomous LLM agent built on the smolagents framework, enabling dynamic decision-making about when and how to retrieve information. During development, the engineer encountered significant challenges running the model locally on CPU, including import errors, latency exceeding 213 seconds per step, and model hallucinations. These issues were resolved by switching to a cloud-hosted model, Qwen2.5-72B-Instruct, and applying dynamic score normalization with a weighted fusion parameter. The research also outlines a future roadmap for scaling the architecture toward knowledge graphs, multi-agent consensus systems, and automated valuation frameworks.