SShortSingh.
Back to feed

Structured file design, not better prompts, fixes RAG hallucinations on legal dates

0
·1 views

A team at embedded IoT security firm Platanor found that AI models, including ChatGPT, consistently confused the EU Cyber Resilience Act's entry-into-force date (2024) with its actual application deadline (2027) when processing raw regulation PDFs. The root cause was identified not as model failure but as poor source structure: token-based chunking split articles mid-sentence, and dates appeared across documents with no explicit contextual links. To address this, the team restructured their reference base by chunking text at natural article boundaries, embedding source-priority rankings directly in each file, and adding explicit verification dates alongside critical deadlines. They also added an llms.txt index at the repository root so AI agents could selectively load relevant files rather than parsing the entire document corpus. The team has published their fact-checked regulatory reference base covering CRA, RED, NIS2, and CSA as an open repository compatible with custom RAG pipelines and Claude Skills.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

RAG vs. Full-Context AI Reading: A Developer Tests Both on Real Documents

A developer built an open-source pipeline to compare two AI answering approaches — Retrieval-Augmented Generation (RAG) using BGE-M3 and Qwen3, versus direct full-document context — on a research paper and a full-length book. The pipeline was designed to be deliberately basic, using fixed-size chunking and plain cosine similarity, to expose where standard RAG setups fail before any optimizations are applied. On a SIGUL 2024 paper about English-Nepali legal machine translation, both methods produced accurate answers, though the direct-context approach returned more precise numeric detail. However, when tested on the book 'Hands-On Large Language Models,' the RAG method retrieved irrelevant chunks and returned a completely incorrect summary, while the full-context approach correctly identified the document. The experiment highlights a key weakness of vanilla RAG: retrieval quality is heavily dependent on chunk relevance, and without reranking or smarter chunking, the system can confidently return wrong answers.

0
ProgrammingDEV Community ·

AWS WAF Challenge used to block large-scale bot attacks at the network edge

A web security consultant was called in after a client's login page endured a week-long bot attack generating millions of requests from a vast number of IP addresses, making IP-based blocking ineffective. The attackers also rotated JA3 and JA4 fingerprints and industrialized token acquisition, partially bypassing an existing Cloudflare Turnstile integration by solving challenges in one country and replaying tokens from another. Because Turnstile validation occurred deep in the application stack, each rejected request still consumed CDN, load balancer, PHP, and database resources, driving up costs and degrading performance at scale. The consultant deployed AWS WAF's Challenge feature, which intercepts requests lacking a valid token at the network edge, before they reach any application infrastructure. This approach was applied across two attack scenarios — a legacy server-side HTML app and a modern single-page app calling a JSON API — using both integration modes offered by AWS WAF Challenge.

0
ProgrammingDEV Community ·

One Unplugged Cable Took Down Whole-Home DNS, Exposing a Hidden Single Point of Failure

A home network in France appeared to lose internet access entirely after the owner accidentally unplugged the wrong cable, disconnecting not his NAS but his local DNS server. All devices — phones, laptops, and TV — showed no connectivity, yet the internet connection itself was fully functional throughout. Layered network testing quickly revealed that the router and raw IP access worked fine, but name resolution had completely failed. The outage stemmed from a self-hosted split-horizon DNS setup, where all internal subdomain resolution depended on a single machine with no reliable failover. Although a secondary DNS was configured, devices had not been set up to switch over quickly, illustrating that untested redundancy offers little real protection.

0
ProgrammingDEV Community ·

AWS EC2, Load Balancing, and Auto Scaling: Core Compute Concepts Explained

Amazon EC2 is AWS's Infrastructure-as-a-Service offering that lets users launch and manage virtual machine instances in the cloud, with full control over the operating system, software, and security. Key building blocks include Amazon Machine Images (AMIs) as launch templates, instance types defining CPU and memory, key pairs for secure access, and security groups acting as virtual firewalls. Elastic Load Balancer distributes incoming traffic across multiple EC2 instances, while Auto Scaling automatically adjusts the number of running instances in response to real-time demand — a concept known as elasticity or horizontal scaling. In contrast, scalability refers to resizing a single instance vertically by changing its instance type, which requires stopping the instance and is considered a long-term capacity decision. Together, ELB and Auto Scaling form the foundation of resilient, cost-efficient application architecture on AWS.