SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

ServiceNow Releases EVA-Bench Data 2.0 with 213 Voice Agent Test Scenarios Across 3 Domains

0
·1 views

ServiceNow AI Research has launched EVA-Bench Data 2.0, an open-source benchmark designed to evaluate enterprise voice agents across airline customer service, IT service management, and healthcare HR service delivery. The updated benchmark covers 213 evaluation scenarios and 121 tools — approximately four times the scope of its predecessor. All scenarios were derived from real phone-based customer service workflows and validated for solvability by three leading AI models: OpenAI GPT-4.5, Google Gemini 3.1 Pro, and Anthropic Claude Opus 4.6. The healthcare domain notably incorporates domain-specific regulatory details such as NPI provider identifiers, FMLA regulations, and insurance coverage rules to reflect real-world complexity. The datasets are freely available on Hugging Face, with a multilingual expansion planned for a future release.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Four AI frontier models launched in four days as simultaneous API outages expose single-provider risk

Four major AI models — Claude Fable 5.1, Gemini 3.8 Flash, Meta Muse Spark 1.3, and OpenAI GPT-6 Astra — were released within a four-day window in early September, signaling that the frontier model release cycle has compressed from quarters to days. That same week, OpenAI, Anthropic, and Grok services went down simultaneously, sparking a widely discussed thread on Hacker News highlighting how much shared infrastructure underpins the AI API economy. The rapid release cadence imposes a recurring 're-evaluation tax' on developers, who must repeatedly assess whether to migrate as benchmarks, pricing, and tool behavior shift with each new model. Analysts and developers argue that relying on a single AI provider now creates both availability and scheduling risk, not merely a contingency problem. The recommended response is to build abstraction layers — combining a local model for routine requests with a routing layer for frontier tasks — so that switching providers becomes a low-cost, routine decision rather than a disruptive one.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

South Korea Launches Free Nationwide AI Access for 52 Million, Mandates 80% Domestic Models

South Korea's Ministry of Science and ICT has finalised three consortiums — SK Telecom, KT, and Kakao — to deliver its 'AI for All' programme, offering all 52 million citizens free generative AI access with no declared token limits. A beta rollout is planned for September 2024, with a full national launch by year-end. The government has allocated 512 NVIDIA B200 chips to support the initiative, with national budget funding set to partially cover operating costs from 2027 onward, making South Korea the first G20 nation to undertake such a programme. A key regulatory condition requires operators to route at least 50% of queries through certified Korean sovereign AI models and an additional 30% through other domestic models, setting an 80% floor for homegrown AI usage. Analysts note the policy functions less as a public welfare measure and more as industrial strategy, using mass citizen traffic to build sustained demand for South Korea's domestic AI ecosystem.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Why RAXXO's Solo Developer Treats Semantic Versioning as a Non-Negotiable Rule

RAXXO, a solo software studio, enforces strict semantic versioning across all its tools, using major, minor, and patch numbers with precise, unchanging definitions. A patch release fixes bugs only, a minor release adds features without breaking existing ones, and a major release signals intentional breaking changes with prior warning. The developer learned this discipline the hard way after a silent breaking change was mislabeled as a patch, triggering avoidable user support issues. Version numbers are paired with changelog entries for every release, as neither is considered useful without the other. Beyond user trust, the system also serves as a personal memory aid when switching between multiple codebases that may not have been touched in weeks.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Why Flask's Dev Server Can Work as a Desktop App Backend If Done Right

Flask's built-in development server carries a warning against production use, but desktop applications operate under fundamentally different conditions that can make it a viable runtime choice. Unlike public web services, a desktop app's Flask server binds only to the local loopback address (127.0.0.1), meaning it is inaccessible from external networks and serves just one user on the same machine. However, developers must explicitly disable Flask's file-watching reloader, which spawns two processes and can break single-instance checks and port management in packaged apps. Enabling threaded mode (threaded=True) is also essential, since the dev server otherwise handles requests sequentially, which can cause stalls when multiple requests — such as a progress stream and a heartbeat ping — run concurrently. Accidentally binding to 0.0.0.0 instead of 127.0.0.1 would expose the server to any device on the local network, undermining the core safety assumption that makes this approach acceptable.

0 comments Read more at DEV Community