Study: AI writes Ruby well but struggles to navigate large, complex codebases

·2 views

A developer tested AI agents across 13 real Ruby codebases to assess whether Ruby and Rails are truly the most AI-friendly stack. The research distinguished between two skills: reading code locally versus navigating structural dependencies across files. In small, well-organized Rails repos, AI agents performed strongly even without extra tooling, supporting the stack's AI-friendly reputation. However, in larger, more complex codebases with scattered dependencies and polymorphic patterns, baseline AI recall dropped as low as 0.24 without a structural map to assist navigation. The findings suggest the claim holds for writing and reading Ruby, but breaks down for dependency navigation at scale.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

China-Linked Hackers Target Indian Tax Filers With Fake ITR Tools to Deploy Malware

Cybersecurity firm Seqrite Labs has uncovered a campaign, dubbed Operation DragonReturn, in which a China-nexus threat group distributes DcRAT — a remote access trojan — by impersonating India's Income Tax Department. Attackers use phishing emails and malicious links disguised as legitimate tax filing utilities, timed to coincide with India's March 31 fiscal deadline to maximise success rates. Once installed, DcRAT establishes persistence on infected systems and harvests sensitive data including credentials, browser history, and keystrokes, relaying it to attacker-controlled servers. The campaign specifically targets Indian tax professionals and corporate finance teams, who typically hold elevated access to financial records, making them valuable entry points for deeper network infiltration. The use of commodity malware-as-a-service infrastructure suggests the attackers sourced DcRAT from Chinese underground forums, where such tools are actively maintained and updated.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Supracorona Login Gate Plugin Offers Simple Site-Wide Access Control for WordPress

A developer has released Supracorona Login Gate, a lightweight WordPress plugin designed to restrict entire websites to logged-in users only. The plugin targets use cases such as staging environments, client portals, internal tools, and private documentation sites that do not require full membership systems. When enabled, logged-out visitors are redirected either to the standard WordPress login page or a custom page chosen by the site administrator. Unlike bloated membership plugins, it focuses solely on one question: whether the current visitor is authenticated. The plugin is now publicly available on WordPress.org.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How to Performance Test RAG Apps for Speed and Answer Quality in CI/CD

A technical guide published on DEV Community outlines a dual-gate approach to performance testing Retrieval-Augmented Generation (RAG) applications. Traditional load testing tools like k6 can measure response times but fail to detect hallucinations, where a model returns fast yet factually incorrect answers. The guide recommends pairing k6 for latency metrics — including Time to First Token and inter-token latency — with DeepEval, which uses an LLM-as-judge method to score faithfulness and answer relevancy. Both testing layers are integrated into a GitHub Actions CI/CD pipeline, enabling automatic regression detection on every pull request before code reaches production. The approach addresses a core weakness in conventional API testing: a RAG system can appear performant while silently generating unreliable, fabricated responses.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer builds AI memory auditor to verify data deletion under GDPR compliance

A developer built a tool called Lethe during a Cognee hackathon to test whether AI systems can cryptographically prove they have forgotten a user's data after deletion requests. The project was motivated by legal obligations under GDPR Article 17 and India's DPDP Act 2023, both of which grant users a right to erasure, with European regulators making enforcement a priority in 2026. Lethe uses an automated auditor agent that fires 15 standardised probes at an AI memory system before and after deletion, scoring each response as either a data leak or safe. Testing revealed that standard record deletion left residual references to a user embedded in other customers' data within the knowledge graph, producing an incomplete erasure verdict. A full cascade deletion that redacted all cross-references across the graph reduced leaked probes from 15 to zero, generating a verifiable erasure certificate.

0 comments Read more at DEV Community