Developer launches AgentEval Forge, an open-source agent evaluation harness on PyPI

·1 views

A developer has publicly released AgentEval Forge, an open-source agent evaluation framework now available on PyPI, built to assess AI agents beyond simple answer quality. The tool evaluates entire agent runs, including tool usage, execution paths, and safety boundaries. It features five framework adapters, 17 deterministic scorers, 11 LLM-as-judge metrics, and a security model with sandbox mode and audit trails. The developer field-tested the harness against 19 real GitHub agents across LangGraph and PydanticAI frameworks, screening over 150 repositories to find viable candidates. Real-world integration testing revealed significant gaps that unit tests and mock agents had masked, shaping the tool's final architecture across 12 milestones and 118 tasks.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

How to Call Real Phone Numbers from a Browser Using WebRTC and SIP

A developer guide explains how to connect browser-based WebRTC audio to the public telephone network (PSTN) using a SIP telephony provider such as Telnyx. The architecture involves three components: the browser capturing audio via WebRTC, a telephony provider bridging WebRTC to traditional phone infrastructure, and a backend server that issues short-lived credentials. A key security rule emphasized throughout is that API keys must never reach the browser — only a JWT token of limited duration is sent to the client. The guide also highlights non-obvious billing and timing quirks, such as zero-second calls appearing on invoices and CDR rounding mismatches. Virtual phone numbers can be purchased and configured programmatically via REST API as part of the same setup.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

AI Chat Assistants Vulnerable to Transcript Injection via Browser DevTools

A security flaw affecting many production AI chat assistants allows attackers to manipulate conversation history directly from a browser's network tab, requiring no prompt engineering or special tools. The vulnerability stems from a common implementation pattern where the client sends the entire message history to the server on every turn, meaning all messages — including those claiming to be from the assistant — are effectively user-controlled input. By injecting fabricated assistant turns, an attacker can make the AI model believe it has already verified a user's identity or granted elevated permissions, bypassing content filters entirely since no suspicious language is used. The root cause is that AI models treat assistant-role messages as trusted records of what actually occurred, making forged history more convincing than traditional jailbreak prompts. The fix is straightforward: servers should retrieve conversation history from their own database rather than accepting it from the client, accepting only the latest user message per request.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Recreates Jollof Rice in Pure CSS for Frontend Art Challenge

A developer named Eze-Charles submitted a CSS art piece depicting a bowl of jollof rice as part of DEV Community's Frontend Challenge - Comfort Food Edition. The artwork was built entirely with CSS, using no images or SVG assets of any kind. Rice texture was achieved through layered radial gradients at varying sizes, while steam effects were created using four blurred circles. The creator noted that the steam animation was the element they were most proud of in the project. Looking ahead, the developer plans to attempt a second CSS dish featuring liquid movement or a pour animation.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Five Major Dev Tools Released This Week: GPT-5, Copilot X, Next.js 15 and More

Several significant developer tools launched in quick succession between July 27 and July 31, 2026, spanning AI models, frameworks, and cloud utilities. OpenAI released GPT-5 on July 29, 2026, alongside a new version 2.0 of its Python SDK, both aimed at improving how developers build AI-powered applications. GitHub followed on July 31 with Copilot X, an enhanced AI coding assistant featuring updated workflow integration and code assistance capabilities. React's Next.js framework reached version 15 on July 30, bringing performance improvements and new features relevant to frontend developers. Rounding out the week, Amazon released AWS CLI v3 on July 27, introducing performance gains and new commands for developers managing cloud infrastructure.

0 comments Read more at DEV Community