Benchmark Reveals PMapper Crashes on Timeout Errors, cloudfox Runs Cleanly on AWS IAM Lab

·1 views

A developer tested two open-source AWS IAM privilege-escalation tools — PMapper 1.1.5 and cloudfox 2.0.5 — against a deliberately misconfigured AWS account over two weekends. PMapper failed to complete its graph-building step because its error-handling code catches region refusals but not connection timeouts, causing the tool to crash and discard four minutes of already-collected data. The tool also does not run on Python 3.10 or later despite its documentation claiming support from Python 3.5 onward, with the underlying incompatibility unaddressed on the main branch since early 2022. cloudfox, by contrast, completed its scan in under a minute, producing detailed CSVs covering 94 principals and over 6,000 permission grants with no errors. The findings highlight how subtle gaps in error handling and outdated dependency support can render security tools unreliable in real-world AWS environments.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Why Language Models Should Not Be Trusted to Extract Numbers From Documents

A technical analysis warns that language models pose a hidden risk when used to extract numerical data from documents, as numeric errors — unlike missing fields — can go undetected for months. Unlike an empty field, which prompts human review, a plausible but incorrect number such as 340,000 in place of 349,000 gets forwarded, quoted, and copied without scrutiny. Research indicates that language models process multi-digit numbers digit by digit, making errors that are close in string format but vastly different in actual value. Structured output schemas, including those offered by providers like OpenAI, guarantee field format compliance but explicitly do not guarantee that the extracted value matches the source document. The author argues that treating a model as a reliable document reader is an unverified assumption, and that a system leaving some fields empty while being accurate on the rest is more trustworthy than one that fills all fields with occasional silent errors.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Tool Misuse in AI Agents: How Behavioral Standards Can Prevent Costly Errors

AI agents managing large sets of enterprise tools frequently call the wrong function due to overlapping descriptions and lack of behavioral context, sometimes triggering destructive actions like deleting users instead of deactivating them. As tool libraries scale from a handful to hundreds, these mismatches become increasingly risky and difficult to catch through prompt engineering alone. A proposed open standard called apcore addresses this by requiring structured schemas and behavioral tags — such as marking operations as destructive — before a tool can be registered. The framework also introduces self-healing error guidance, where a structured response directs the agent to correct its mistake and retry autonomously rather than crashing or looping. The core argument is that reliable agentic systems require enforceable engineering standards, not just better prompts.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

GLM-5.3-Flash Priced at 1/40th of Claude Opus 4.8, Targets AI Agent Workloads

Zhipu AI's GLM-5.3-Flash, a 320-billion-parameter open-source model, is priced at $0.30 per million input tokens and $1.20 per million output tokens internationally — roughly one-fortieth of Claude Opus 4.8's official list price. The model ranked first on OpenRouter under the anonymous name 'Ox Alpha' on its launch day and processed approximately 62 trillion tokens within its first six days. GLM-5.3-Flash supports a 1.04-million-token context window and native multimodal inputs including text, images, video, and files, placing it in the same low-cost tier as DeepSeek V4 Flash while scoring higher on the Artificial Analysis Intelligence Index. Analysts note the 1/40th cost ratio compares against Opus 4.8's public list price, not enterprise-negotiated rates, and that per-task costs depend on how many retries or additional calls a workload requires. The model's low input-token pricing is seen as particularly advantageous for long-context agentic applications where repeated context re-reads dominate overall API spend.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

AI-Generated Rust Code Passed Tests but Hid a Subtle Concurrency Bug

A software developer asked an AI model to write a lock-free single-producer single-consumer ring buffer in Rust, and the generated code compiled cleanly and passed unit tests. However, the code contained a memory-ordering flaw: all atomic operations used Relaxed ordering, which on architectures like ARM provides no guarantee that a data write becomes visible to a consumer before the updated head pointer does. This creates a torn read, where the consumer sees a new index but fetches uninitialized or stale data from the buffer slot. The fix requires using Release ordering on the producer's store and Acquire ordering on the consumer's load to enforce the correct visibility sequence. The author argues this illustrates a core risk with AI-generated unsafe Rust: the model reproduces patterns from training data without understanding the memory-model invariants that determine whether those patterns are actually safe.

0 comments Read more at DEV Community