Study: LLM-generated AGENTS.md files hurt task success and raise AI costs

·1 views

A study comparing developer-written and LLM-generated AGENTS.md instruction files found that the generated versions underperformed in five out of eight test settings. The AI-generated files added unnecessary steps per task and increased inference costs without improving output quality. Researchers attribute this to models filling instruction files with generic best practices rather than project-specific context they cannot know. Details such as critical directory restrictions, hard-learned conventions, or precise command flags can only come from the developers themselves. Experts recommend using LLMs only to scaffold a skeleton structure, with humans writing and pruning the substantive content.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

How Tracking AI Research Today Can Shape Smarter Product Decisions Tomorrow

Academic AI research typically takes two to four years to translate into widely used product features, creating a strategic window for product managers and business owners who pay attention early. Institutions like Berkeley's AI research group are currently focused on areas such as reliable reasoning, physical-world interaction, human-AI collaboration, and predictable system behavior. Experts suggest that practitioners do not need deep technical knowledge, but rather a broad awareness of which problems researchers are actively solving. For example, a content business tracking generative modeling and reasoning research could anticipate that AI tools will soon handle more execution tasks, shifting human value toward strategy and creative direction. Building simple habits — such as following research blogs and monitoring announcements from companies like Google, Anthropic, and OpenAI — can help professionals adjust their positioning before market shifts force them to.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Node-RED on Industrial Edge Gateways Simplifies Factory IoT Data Workflows

Industrial IoT projects often require a bridge between factory-floor equipment and upper-layer systems, a role filled by industrial edge gateways like the Robustel EG5120. Node-RED, a lightweight flow-based tool, can run on such gateways to handle local data processing before information is sent upstream. A common use case involves reading raw Modbus device values, transforming them into structured MQTT messages with labels, units, and timestamps. The Robustel EG5120 supports this architecture by providing field connectivity, local computing, network management, and remote deployment capabilities. However, the effectiveness of any such workflow depends on project-specific configuration, installed nodes, data sources, and security planning.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

GraphRAG Outperforms Standard RAG for Enterprise Social Listening, Developer Argues

A developer writing on DEV Community argues that GraphRAG — which combines large language models with knowledge graphs — addresses key limitations of standard retrieval-augmented generation in enterprise AI. The argument is partly inspired by a research paper on dataset discovery, which demonstrated how graph-based architectures can generate explainable, relationship-aware results. Applied to social listening, GraphRAG can trace causal chains — such as how an influencer's criticism of a product feature triggered a broader negative trend — rather than simply returning text chunks that match keywords. Unlike flat vector search, graph traversal allows AI systems to link market signals, consumer complaints, and macroeconomic context into a coherent narrative with a transparent data trail. The author plans to continue the series across multiple industries and will release open-source proof-of-concept tools, starting with a GraphRAG social listening platform.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Why listing exact CLI commands in AGENTS.md boosts AI coding agent accuracy

Developers using AI coding agents are advised to include precise CLI commands in their AGENTS.md file to improve reliability and reduce errors. Vague instructions like 'run the tests' cannot be executed by agents, whereas specific commands such as 'pnpm test' can be run, read, and acted upon directly. Agents that lack explicit commands may guess incorrectly or falsely report success on code that was never verified. The recommended commands cover installation, development, building, testing, typechecking, and linting, with single-file test commands enabling faster iteration. Aligning these commands with CI requirements ensures that work the agent considers complete will also pass in production pipelines.

0 comments Read more at DEV Community