Opik: Open-Source LLM Observability Tool Lets Teams Self-Host Agent Tracing

·1 views

Opik is an open-source, self-hostable observability platform for LLM agents, first created on May 10, 2023, and actively maintained with over 7,000 commits as of September 2026. The tool allows developers to trace every decision an agent makes before a failure occurs, addressing the common problem of discovering errors only after users complain. Licensed under Apache-2.0, Opik can be deployed entirely within a team's own infrastructure, with no data leaving the system and no enterprise sales process required. Beyond basic tracing, the platform bundles LLM-as-a-judge evaluation, prompt management, RAG assessment, guardrails, and CI/CD integration via PyTest into a single suite. Coding agents such as Claude Code, Cursor, and VS Code Copilot can also connect directly to Opik to read traces and run evaluations from within a chat interface.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

How to Verify Which Python Binary Your Code Actually Runs

Developers often assume commands like 'python' or 'pip' point to the correct interpreter, but this can silently cause errors in scripts and CI pipelines. On many systems, 'python' is either missing or aliased differently from 'python3', making explicit checks essential. Virtual environments add another layer of confusion, as a new shell session may not activate the venv even if the directory exists. Similarly, 'pip' and 'python3 -m pip' can belong to different interpreters, leading to packages being installed in the wrong environment. A short shell audit script using commands like 'command -v', 'sys.executable', and exit code checks can quickly confirm which binaries are actually in use.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

GitHub Copilot Can Now Formally Approve Pull Requests, Counting Toward Branch Rules

GitHub updated its Copilot code review tool on 1 September 2026, allowing it to submit approvals that count toward a repository's required-approvals rule, the same way a human teammate's approval would. Previously, Copilot only left advisory comments and its reviews explicitly did not satisfy branch protection requirements. Administrators must actively enable the feature, and the approval is automatically dismissed if new commits are pushed, mirroring existing human-review behaviour. A follow-up update on 11 September added shell-tool execution during reviews, auto-resolution of Copilot's own comments, and an ensemble of agents that GitHub says improved detection of high-severity issues by 47%. The changes are currently in public preview and subject to change, raising governance questions about how AI approvals interact with enterprise and repository-level branch protection policies.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer flags self-asserted test steps and untestable surfaces in E2E run

A developer running end-to-end tests on a sandbox project reported a 10/10 pass rate, but noted that one of the ten steps could not be externally verified — it was explicitly labelled as self-asserted rather than silently marked green. The untestable property concerned whether the binary correctly refuses unknown operation names, a check that is impossible to automate because the command interface has no argument slot through which a bad operation name can be passed. The developer argues this is a product observation, not a test limitation: if something cannot be tested from outside, it often also cannot be used from outside, and both gaps share the same fix. The remaining nine steps were validated across two independently created sandboxes, with all 46 output lines matching byte-for-byte, made possible by deriving environment identifiers from content rather than random allocation. An additional hour was lost to Git Bash silently rewriting POSIX paths before they reached the program, a transformation that produced no error and caused failures far from the actual source.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Builds Immutable Invoice API Where Corrections Create New Documents

A developer published a TypeScript/Express REST API that enforces a strict invoice lifecycle, making finalized invoices immutable and handling corrections by issuing new replacement documents rather than editing originals. The system uses four states — draft, issued, canceled, and replaced — with replacement operations wrapped in a single database transaction to prevent inconsistent states. Despite achieving over 97% test coverage across 56 tests, two real bugs slipped through undetected by automated testing. One bug involved incorrect currency division that rendered Vietnamese Dong amounts as hundredths of their actual value on generated PDFs, a mistake the tests missed because both the code and expected test values shared the same flawed assumption. The incident highlights a key limitation of unit testing: automated tests confirm code behaves consistently with developer expectations, but cannot catch bugs rooted in incorrect assumptions themselves.

0 comments Read more at DEV Community