RAG Explained: How AI Can Answer Questions Using Your Private Documents

·1 views

Retrieval-Augmented Generation (RAG) is a technique that allows AI models to answer questions using a company's private documents without requiring expensive retraining. The process involves three core steps: splitting documents into manageable text chunks, converting those chunks into numerical representations called embeddings, and using a vector database to find the most relevant chunks when a user asks a question. Developer Ishan Sharma demonstrated the approach by building Guidely, an internal knowledge assistant that retrieves answers directly from company documents. To improve efficiency, Guidely uses a SHA-256 hash-based cache to avoid regenerating embeddings for unchanged content, and relies on Facebook's FAISS library for fast similarity searches across large sets of embeddings. RAG effectively bridges the gap between a general-purpose AI model and an organisation's specific, ever-changing knowledge base.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Developer Builds AI Guardrail System to Block Unsafe or Off-Topic LLM Responses

A developer has published an open-source didactic project demonstrating how guardrails can be applied to generative AI systems to prevent unsafe content, hallucinations, and information leakage. The application allows users to submit a prompt along with optional sensitive topics, which are then used to evaluate whether the AI's response should be blocked. Built on a stack of C#/.NET 10, React 19 with TypeScript, SQLite, and Google's Gemini Flash Lite model, the system uses the LLM itself as a judge in a second API call — a pattern known as LLM-as-judge. If the response touches on any flagged topic, the content is replaced with a 'not permitted' message, and all queries are logged for audit purposes. The developer notes that real-world deployments should use separate models for content generation and judgment to reduce the risk of false negatives.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Builds AI Agent System That Turns Phishing Prompt Injections Into Evidence

A developer built an agentic cybersecurity tool called Sentinel, designed to detect and flag phishing domains by monitoring public Certificate Transparency logs in real time. The system uses a layered pipeline of specialized AI agents — including local and cloud-based language models — to triage suspicious domains at near-zero cost, discarding roughly 99% of candidates before any token expense is incurred. A key discovery during development was that phishing pages sometimes embed prompt injection instructions targeting AI crawlers, and Sentinel was redesigned to treat such attempts as forensic evidence rather than simply stripping them. The architecture enforces strict separation of concerns, with a human approval step required before any irreversible takedown action can be triggered. The project also surfaced a real vulnerability where structured protocol data from RDAP could be attacker-influenced, highlighting that deterministic data sources are not inherently trustworthy.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

AWS g6 Delivers 3.7x LLM Throughput Over g5g Due to GPU Architecture Gap

A developer benchmarked AWS g5g.2xlarge and g6.2xlarge instances serving Google's Gemma-4-E2B-it model using identical code and weights, finding the g6 achieved roughly 3.7 times higher decode throughput. The g5g pairs a Graviton2 host with an NVIDIA T4G (Turing, SM 7.5), while the g6 uses an x86_64 host with an NVIDIA L4 (Ada, SM 8.9) and significantly more GPU memory. Profiling revealed that 87% of decode time on the g5g was consumed by dtype conversion and an fp32 fallback path, because the Turing architecture lacks native bfloat16 support and silently emulates it. The g6's L4 GPU, which natively supports bfloat16, ran near its full memory-bandwidth roofline, while the g5g operated at only 26% of its own. The findings highlight that a mismatched compute dtype causes no errors but silently degrades throughput, making architecture-aware dtype selection critical when deploying LLMs on GPU instances.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Gemma 4 Runs in Pure JAX Across NVIDIA Turing and Ada GPUs With One Codebase

A developer has published a hand-written port of Google's Gemma 4 language model in pure JAX, benchmarked across two AWS spot instances running NVIDIA T4 (Turing) and L4 (Ada) GPUs. The same source code, build ID, and model checkpoint were used on both cards, with the key difference being compute dtype — float16 on the older Turing architecture and bfloat16 on Ada, selected automatically at runtime. The port handles four structural quirks of Gemma 4, including two attention head dimensions, 8:1 multi-query attention, a KV-share map collapsing 35 layers onto 15 caches, and a 4-bit quantized 4.70 GB positional embedding table. A notable finding is that JAX's XLA attention path avoids the shared memory errors that cause vLLM's Triton backend to fail on Turing hardware. However, the Pallas-based fused W4A16 kernel exceeds shared memory limits on both GPUs, meaning neither card benefits from the fast quantized path and both serve the dense 16-bit reference model instead.

0 comments Read more at DEV Community