Two Laravel AI Eval Packages Compared: Pest Plugin vs Vizra Evals

·1 views

Laravel developers testing AI agents now have two packages with overlapping names — pestphp/pest-plugin-evals and Vizra Evals — raising questions about which to use. Both packages share the same --evals flag and PEST_EVALS environment variable, meaning they can coexist in a single test suite without conflict. The Pest plugin offers a broad range of deterministic and model-scored expectations, including consistency testing via repeated prompt runs, making it well-suited for point-in-time quality checks. Vizra Evals, built on top of Pest, goes further by persisting every run to a database, enabling baseline comparisons and row-level regression detection across deployments. The key differentiator is whether a team needs to know if an agent performs well today, or whether it has degraded compared to a previous reference run.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Dark Caracal Deploys GoCaracal Malware Using SVG Phishing and Ethereum Backup C2

Arctic Wolf Labs published research on August 26, 2026, detailing a new campaign by the threat group Dark Caracal, which distributes a Go-based malware called GoCaracal through malicious SVG files disguised as Spanish-language financial and tax documents. Victims are lured via phishing emails into opening SVGs that redirect them through URL shorteners to attacker-controlled sites, where a lightweight GoCaracal executable is delivered inside a 7-Zip archive. Once executed, the malware collects host data, establishes encrypted C2 communication, and deploys an extended build alongside the Delphi-loaded Bandook trojan, enabling browser credential theft, keylogging, SOCKS5 proxying, and WebRTC remote desktop access. Notably, if the primary command-and-control server becomes unreachable, the malware retrieves a backup C2 address stored in an Ethereum smart contract via JSON-RPC calls, making takedown efforts significantly harder. Defenders are advised to block SVG and archive attachments at mail gateways, restrict unauthorized executables, and filter outbound Ethereum JSON-RPC traffic to counter this threat.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Used Claude AI to Cut 8,400 Weekly Errors Down to 11 Real Bugs

A software developer built a 200-line Python pipeline that feeds structured error data and repository context into Claude Code to automatically triage production issues. Their error tracker was generating around 8,400 events per week across roughly 340 distinct issue groups, far exceeding what any engineer could manually review. The top errors by volume were largely noise — bot traffic, browser warnings, and user-aborted network requests — while genuine bugs were buried deep in the list. The AI agent evaluated each error cluster and returned a verdict, flagging 11 real bugs that had gone unnoticed for months, including a null dereference tied to a 2024 database schema change. The approach aimed not to fix bugs automatically, but to replicate the routine four-minute human triage pass at scale, so engineers only spend time on issues that truly matter.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How to Build a Production-Ready Polymarket Arbitrage Bot in Python

A production-grade Polymarket arbitrage bot requires more than checking if two prices sum below $1 — it must account for fees, liquidity, partial fills, and execution risk. The recommended architecture separates market data ingestion, strategy detection, risk management, and execution into distinct layers, preventing the strategy from directly controlling the exchange client. Real-time order book data is consumed via Polymarket's public WebSocket channel using the CLOB V2 trading stack and the official py-clob-client-v2 Python library. Fee-aware opportunity detection compares the combined executable cost of YES and NO legs against the fixed $1 payout, filtering out trades that do not clear a minimum edge threshold. Critical risk controls include position size limits, stale-book timeouts, one-leg fill timeouts, daily loss limits, and emergency shutdown mechanisms to manage two-leg execution exposure.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Dev builds AI sanctions-screening tool for NGOs, uncovers critical error-handling flaw

A developer built an AI agent called Interdict to automate OFAC sanctions screening for small NGOs, which face the same compliance exposure as major financial institutions but lack resources to manage it. In its first real test run using Google's Gemini model, the system quarantined 438 of 536 counterparties — flagging them for human review. The root cause was not a model failure but a coding bug: API rate-limit errors were misclassified as model-integrity failures, flooding the compliance queue with false escalations. This buried genuine cases requiring human attention under hundreds of irrelevant entries, effectively turning a safety signal into noise. The fix involved distinguishing transient network errors from true model failures and implementing proper retry logic with server-guided backoff.

0 comments Read more at DEV Community