Benchmark finds AI coding agents' code often passes tests yet remains exploitable

·1 views

A preprint posted to arXiv in November 2026 introduces A.S.E 2.0, a benchmark of 145 tasks drawn from real CVEs across 51 open-source projects, to measure the security of AI-generated code. Tasks count as secure only when functional tests pass and a working proof-of-concept exploit fails. Claude Code running Sonnet 4.5 passed functional tests on 74.0% of tasks, but only 18.4% of its output was secure, and a majority of test-passing output remained exploitable across all tested setups. The researchers found agents rarely searched for existing security safeguards, and that naming the relevant safeguard in the prompt substantially improved results. The study has not yet undergone peer review and tested only a limited set of models.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

PHP Session Upload Progress Advisory: CVE-2026-115560 Mitigation and Scope

PHP published an informational notice on October 23, 2026, about CVE-2026-115560, which applies only when the session serialize handler is set to php_binary or php_serialize and then changed at runtime to php before session_start() with session upload progress enabled. PHP describes remote code execution through this path as unlikely and recommends disabling upload progress until the next regular bugfix release, while noting that applications that never change the handler at runtime can keep it enabled. A full advisory is promised, but no complete list of affected versions or CVSS score has been published yet, and cPanel has said affected versions remain unknown. Red Hat states the recommended mitigation is fully protective and gives guidance for enforcing it in FPM and Apache. A ZoomEye search for app="PHP" returned about 111 million PHP fingerprint matches, which reflects general PHP deployment rather than confirmed vulnerable systems.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Timeouts Are Not Failed Bookings: Designing Duplicate-Safe AI Actions

A DEV Community article argues that an AI assistant should not assume a booking failed just because a request timed out, since the service may have already saved it. It recommends treating each booking as one durable operation with a unique identifier that excludes private customer data. The author proposes a state model (Prepared, Submitted, Confirmed, Uncertain, Escalated) that separates confirmed failures from unknown outcomes. Examples draw on Google Calendar's client-supplied event IDs and Stripe's idempotency keys, with a warning that each provider's documented guarantees must be checked. The piece advises verifying uncertain writes via supported read paths and escalating ambiguous cases to a person rather than creating duplicates.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Why Email and Phone Normalization Can Go Wrong in Contact Data

A contact lookup can fail when formatting differs, such as spaces in a phone number, and aggressive cleaning may then corrupt valid identifiers. The article stresses that normalization produces a consistent representation, while validation only checks structure and neither proves a person owns an address or number. Email handling should follow a documented domain policy, since providers differ on dots, plus-addressing, and letter case, and stripping plus suffixes can merge identifiers that should stay distinct. Phone numbers without country context are ambiguous, so the guidance is to capture country explicitly, use a maintained parsing library, and keep extensions separate. Original inputs should be retained only under access controls, and uncertain values should trigger review rather than silent guessing.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Safer JSON Snapshot Updates Use a Temporary File Before Replacement

A technical write-up describes how a small local job can avoid corrupting its saved JSON state. Opening the destination file directly erases the prior result before serialization completes, so a mid-write failure can leave readers with a partial snapshot. The proposed approach writes to a temporary file in the same directory, flushes and syncs it, then replaces the original. Three local tests on synthetic files confirmed that successful writes, serialization errors, and replacement failures all leave the previous data intact. The author notes the helper lacks version checks or a writer lock, so concurrent writers could still overwrite each other.

0 comments Read more at DEV Community