JevBench launches open benchmark for typed decision models ranking speed, cost, accuracy

·1 views

A developer has released JevBench, an open-source benchmark designed to evaluate typed decision models — systems that return bounded choices and probabilities rather than free-form text. Unlike large language models, these so-called Jev-class models are claimed to be significantly faster and cheaper while maintaining comparable intelligence on text inputs. JevBench scores models across four weighted dimensions: chance-corrected intelligence, calibration, speed, and cost, using a standardized set of 534 English-language decision prompts. The current leaderboard is topped by Jev at 74.4, followed closely by SemIf, djev, Winnow-12B Q8, and reflex 4B. The project is available on GitHub, though it carries noted limitations including English-only support and potential noise in scores separated by roughly one point.

Read the full story at Hacker News

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Solo Developer Builds Yield, a Free Offline-First Android Task Management App

A solo developer is building Yield, an Android task management app with a free tier called the Basic Unit that focuses entirely on offline, on-device processing. The app lets users capture tasks quickly via OCR, voice input, or manual typing, then organize them into a three-level Todo structure and sync with a calendar. Key free features include on-device OCR for handwritten notes, app-launch detection, Google Maps integration, and Android Auto support, while network-dependent features like AI structuring and Gmail sync are reserved for paid tiers. Todos support cross-linking, so completing an identically named item in one place automatically marks it done elsewhere, using exact string matching to keep the feature cost-free. The app also uses a dedicated file format for sharing tasks between Yield users without requiring AI reprocessing on the receiving end.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Android 17 Adds Native Certificate Transparency: What Developers Must Do Now

Google has shipped Android 17 with built-in Certificate Transparency (CT) enforcement, which verifies during every TLS handshake that certificates appear in a public log. The update is designed to reduce the risk of forged or mis-issued certificates reaching users. Developers do not need to rewrite their TLS stack, as the platform enforces CT automatically for existing libraries like OkHttp and Volley. However, explicit CT support requires updating the compileSdkVersion, adding a network security config XML file, and declaring it in the app manifest. An optional Python audit script and a Kotlin runtime exception handler are also available for developers who want to log failures or apply custom fallback policies.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Ranex Targets Evidence Quality Where GitHub Rulesets Control Channel Access

GitHub Rulesets, editor hooks, and CI pipeline gates are channel controls that govern where code can move — such as during a push, merge, or pipeline stage. These boundaries enforce policy at specific workflow points but do not inherently verify what a passing result actually proves about a given artifact. Ranex takes a different approach by binding evidence to a specific code digest and evaluating whether a claim is genuinely supported, returning a hard failure rather than a skip when evidence is absent. The two approaches address separate questions: channel controls decide whether an action is permitted, while evidence controls determine whether a result is traceable and auditable against the exact code under review. The article argues both layers can coexist, with existing workflow gates retained and evidence requirements layered on top to ensure results remain meaningful and auditable over time.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Android 17 enforces Certificate Transparency, requiring app updates for Play Store compliance

Google's Android 17 introduces native Certificate Transparency (CT) validation, meaning the OS will verify that every TLS certificate is logged in public CT registries before allowing a secure connection. Each certificate must contain at least one valid Signed Certificate Timestamp (SCT) from recognized logs such as Google CT Log, DigiCert, or Let's Encrypt. Apps that are not updated to support CT risk being rejected on Google Play, making compliance a practical necessity for developers. Developers can enable CT support by updating the Network Security Config XML file and referencing it in the AndroidManifest, with optional runtime verification available through the new CertificateTransparency API in Kotlin or Java. The change also helps apps align with regulatory frameworks including eIDAS and PCI-DSS, strengthening the overall TLS trust chain on Android devices.

0 comments Read more at DEV Community