Dev Team Builds AI Agent Observability Tool AXRAY Using SigNoz and OpenTelemetry

·1 views

A development team building an autonomous AI coding agent called AXRAY found they had no visibility into why agent runs varied wildly between 8 and 45 seconds. To solve this, they integrated SigNoz as an observability backbone, using OpenTelemetry and official GenAI Semantic Conventions to instrument every agent action, from LLM calls to Docker command execution. Each agent turn is tagged by phase, allowing the team to separately measure time spent on LLM reasoning versus actual code execution in the sandbox. Analysis of real sessions revealed that roughly 78% of latency originated inside the LLM due to large context windows, not slow infrastructure. SigNoz's ClickHouse-backed trace storage also enabled custom SQL queries, giving the team granular, sub-turn latency breakdowns that standard dashboards could not provide.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Tool actlint audits MCP server safety labels, flags inconsistencies across 31 servers

A developer tool called actlint has been released to check whether safety annotations on MCP (Model Context Protocol) servers accurately reflect what their tools actually do. The tool scans a server's advertised tool list and compares each tool's name, description, and input schema against its declared safety labels, without ever executing the tools. When tested against 541 tools across 31 widely used MCP servers — including those from Stripe, GitHub, Microsoft, Google, MongoDB, and PayPal — only 11 servers came back fully clean. Four servers were found carrying labels that contradict capabilities their own schemas describe, a problem that can silently bypass safety confirmation steps in MCP clients. actlint can be run locally via a single npx command and supports CI integration, offline manifest replay, and baseline tracking to manage existing findings gradually.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How one invoicing app built five AI features without touching its legal records

Slate, an invoicing and bookkeeping SaaS built for Israeli businesses, has integrated five AI-powered features into its platform. The app uses Angular, Express, Prisma, and Postgres, and issues documents that serve as legally binding fiscal records under Israeli law. Israeli regulations require these records to be stored for seven years, numbered sequentially without gaps, and never altered after issue. These legal constraints directly determined how each AI feature was designed and scoped. The five features include receipt scanning, a books-based chat assistant, a support reply drafter, natural language document creation, and smart column mapping for Excel imports.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

AI Safety Guardrails Found to Fail Consistently in Non-English Languages

Research covered by Dark Reading reveals that large language model safety guardrails, mostly trained on English-language data, fail to block jailbreak attempts when the same prompts are submitted in other languages such as French, Polish, or Finnish. The core issue is that refusal training and red-teaming efforts have historically concentrated on English attack patterns, leaving the safety layer undertested for other languages. No specific breach or victim has been reported; the finding highlights a structural vulnerability rather than a confirmed incident. This gap is especially significant for multilingual environments like the EU, where 24 official languages are in use, meaning attackers may need nothing more sophisticated than a translation tool. Semantic embedding-based approaches, which evaluate meaning rather than surface-level text patterns, are cited as a more language-agnostic defense compared to traditional keyword or regex filtering.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

JWT Authentication Explained: How Tokens Replace Passwords After Login

JWT (JSON Web Token) is a widely used authentication method in modern backend development that eliminates the need to repeatedly send passwords with every server request. When a user logs in with valid credentials, the server generates a JWT and sends it back to the client, which then includes it in all subsequent requests. The token consists of three parts — a header describing the algorithm, a payload carrying user details like ID and role, and a cryptographic signature that detects any tampering. Because the signature invalidates the token if it is altered, the server can trust its authenticity without maintaining server-side sessions. This stateless design makes JWT particularly well-suited for REST APIs and applications that need to scale across multiple servers.

0 comments Read more at DEV Community