Building a Snowflake AI Agent Revealed Flaws in Both the Code and Its Evaluation

·6 views

A developer working with a Snowflake Cortex Agent discovered that poor evaluation scores stemmed not only from agent errors but also from flawed tests and misleading metrics. An object the agent failed to find actually existed in the metadata, but required fixes at two levels: updating the agent's fallback instructions and clarifying the semantic view's SQL-generation guidance. Further investigation revealed that a missing tool-call counter was defaulting to zero, making absent telemetry appear as measured inactivity rather than a data gap. The developer maintained three parallel concerns simultaneously — the agent itself, the evidence of its behavior, and the tests used to judge it. The experience highlighted that misdiagnosing whether a failure lies in the data, the tool, or the agent's prompting leads to entirely different and potentially incorrect fixes.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Go Goroutines vs Java 21 Virtual Threads: How They Compare on Memory and Speed

Go's goroutines and Java 21's virtual threads both enable launching hundreds of thousands of concurrent tasks without exhausting CPU or memory, but they achieve this through fundamentally different designs. Go uses an M:N scheduler (the G-M-P model) where goroutines start with a 2KB dynamic stack, allowing 50,000 goroutines to run in roughly 160MB of RAM. Java 21 virtual threads, introduced via Project Loom, store stack frames as heap objects and mount them onto carrier threads only when executing, consuming around 195MB for the same workload — while older Java 17 platform threads crashed at just 4,200 threads. A key behavioral difference is preemption: Go forcibly interrupts goroutines running longer than 10ms via OS signals, whereas Java virtual threads are cooperative and can be monopolized by tight CPU loops. In throughput benchmarks, both runtimes performed nearly identically, with Go at roughly 48,200 requests per second and Java 21 close behind at 46,800.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Silent failures: three bugs that returned success but did nothing

A developer building a business verification API encountered three consecutive bugs that all reported success while silently failing to function. A South Korean government API returned HTTP 200 with empty results when given a wrong parameter value, meaning it would have falsely confirmed no sanctions for every company queried. A scheduled Cloud Run job displayed a green checkmark each time it ran, but a one-line path comparison error on Linux prevented the actual code from ever executing. A third bug caused curl in Git Bash to mangle Korean UTF-8 characters in URLs, returning zero search results and nearly triggering a deep investigation into the wrong part of the system. The incidents highlight a costly and underappreciated failure mode: errors that look like success, spanning API design, deployment platforms, and developer tooling.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Fortinet Patches CVE-2026-26084 Flaw Letting Unauthenticated Users Access FortiSandbox Data

Fortinet disclosed CVE-2026-26084 on September 8, 2026, a high-severity information disclosure vulnerability rated 8.9 affecting multiple versions of FortiSandbox, FortiSandbox Cloud, and FortiSandbox PaaS. The flaw, classified as CWE-284 (improper access control), allows unauthenticated attackers to send crafted HTTP requests and retrieve sensitive device data such as configuration, sample metadata, and analysis logs. The vulnerability stems from the web interface failing to validate requests against an authorized session before returning data, with exposure also depending on how NAT rules are configured at the network edge. Fortinet's internal product security team discovered the flaw, and no exploitation in the wild has been reported. Users are advised to upgrade to FortiSandbox 4.4.9 or 5.0.6 and ensure management interfaces are not reachable from untrusted networks.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How Async Job Queues and Stateful UX Can Fix Broken Social Media Import Pipelines

A development team lost 40,000 Instagram posts after a silent rate-limit error caused their synchronous import pipeline to fail without alerting users. The core problem is that most content import features rely on direct HTTP requests, which are vulnerable to timeouts, API throttling, and partial failures that leave databases with corrupted or incomplete records. The proposed fix involves decoupling the browser session from data ingestion by treating every import as an asynchronous, resumable background task managed by a job queue. A persistent UI drawer subscribes to real-time status updates via WebSockets or Redis-backed polling, so progress is preserved even if the network drops or an external API throttles requests. When slowdowns occur, users are shown an informative banner with an estimated completion time rather than a frozen spinner or a blank screen.

0 comments Read more at DEV Community