Five open science challenges let hidden verifiers, not submitters, score AI-proposed answers

·3 views

FINAL-Bench is running five open science challenges that separate proposing an answer from judging it. Participants submit items such as drug candidates, lithium electrolyte crystals, superconductor pairing estimates, quantum energies or decoders, and a verifier they cannot see scores each one. Answer keys and scoring implementations stay private on the organiser's hardware, while rules and weights are published, so entries cannot be tuned to a public grader. Challenges are open on staggered deadlines, and a LEADBOARD of 21 boards provides starter prompts and skeletons so no special hardware is needed. AI coding agents are explicitly allowed to participate. All scores are computational predictions rather than laboratory measurements, so high scores are hypotheses to be tested.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Automating Approval Flows in Node.js and Postgres with State Machines

Business approvals often stall in email inboxes, leaving no clear record of status, ownership, or deadlines. This guide proposes storing each request in Postgres with a single explicit state, enforced by a CHECK constraint. Every state change is recorded in an append-only event log that supports later audits. Transition rules are kept as plain data in the codebase so the process is easy to review. A scheduled job can then find requests past their deadline and escalate them to the next approver.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

sitetoolsbox.com Combines Browser Tools and Server Checks in One Directory Architecture

A DEV Community article describes how the sitetoolsbox.com platform pairs a navigation directory with browser-based and server-assisted tools. Its client-side JSON formatter runs locally without uploading data and loads only when a user opens it. For bulk domain checks, the server uses RDAP, a JSON-based protocol, with parallel requests via PHP's curl_multi, falling back to WHOIS when needed. The directory is built with static-style HTML, structured data, and internal linking across categories, and empty category pages are excluded from the sitemap. The author argues that keeping the directory's JavaScript lightweight and loading each tool's code only on demand is essential to good performance.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Why an AI write approval needs two timeout windows, not one

The article argues that a single timeout cannot answer two distinct questions when an AI requests permission to write and a person must approve it: whether the conversation should keep waiting, and whether the request can still be decided at all. The author describes a bug where two places computed the offline expiry boundary differently, so the pending list showed rows as actionable that had already expired, causing clicks to fail. The proposed fix separates a short in-conversation wait (60 seconds by default) from a long offline decision window (24 hours by default), and routes every boundary calculation through one shared function. It also addresses races between the in-conversation button and the pending list by using an atomic conditional database update that only succeeds if the status is still pending.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Capturing the request path to debug a 404 from the OpenAI Node SDK baseURL

A 404 from an SDK call does not reveal which URL the client actually requested, so the author suggests first checking the exact request path. The tutorial runs the OpenAI Node SDK against a local HTTP fixture on the developer's machine, requiring no model account or real API key. The fixture accepts only /v1/chat/completions and records the method, path, and model field for each request. Five baseURL configurations are tested, including the common mistake of including the full endpoint in baseURL. The author notes that a fixture 404 only shows a route mismatch in the test setup, not a provider outage or unsupported model, and tested with Node.js v26.0.0 and openai 7.38.0.

0 comments Read more at DEV Community