TypeSafe's Jev Model Scores Mid-Pack Chess Elo at $0.0015 Per Game

·95 views

TypeSafe's Jev is an unconventional AI model that accepts structured inputs with predefined label options and returns probabilistic classifications, rather than generating free-form text. A developer tested Jev on the LLM Chess benchmark by framing each move selection as a Choice query over all legal UCI moves, allowing it to participate in the same evaluation harness used for standard chat models. Jev achieved an Elo rating of approximately 243, placing it near mid-tier reasoning models like o4-mini-medium, while completing games in around 36 seconds at roughly $0.0015 each — far cheaper than its leaderboard neighbors. The model maintained a consistent 50% draw rate against a chess engine across three difficulty levels, with zero illegal moves or protocol failures, highlighting the reliability of its typed, constrained output format. TypeSafe itself does not pursue public benchmarks, so this represents an independent external evaluation showing Jev performs well on tasks involving selection from a fixed list of options.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Connector design should expose partial failures so agents can tell empty results from errors

A connector returning an empty array can signal several different outcomes, including a successful query with no matches, a request that failed before reaching the service, a partially loaded result, or unparseable data. The article argues these should be distinguished, since an agent cannot safely decide its next step if all of them appear as an empty list. It proposes returning results as structured reports that state the status (complete, empty, partial, retryable error, or action required), the source, the fetch time, and any continuation cursor or warnings. Continuation cursors should be saved only after a page is accepted, so an interrupted pagination can resume rather than presenting an incomplete list as complete. For write operations, the author recommends idempotency keys and recording whether an operation was confirmed, since a timeout after a write leaves the outcome ambiguous.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

AWS Organizations: Using Multiple Accounts, OUs and Service Control Policies

The author extends their AWS setup beyond a single account, arguing that separate accounts act as strong boundaries for security, cost and blast radius. The guide explains key terms such as organizations, management and member accounts, OUs, and service control policies (SCPs). It notes that accounts roll up to one consolidated bill, that SCPs only restrict permissions and never grant them, and that they do not apply to the management account. Suggested steps include creating an organization, enabling MFA on the management root user, and setting up a Sandbox OU with a deny policy. Common mistakes to avoid include running workloads in the management account and reusing email addresses across accounts.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How Caching Cut a Product Page's Load Time from Three Seconds to Three Milliseconds

A product page that took three seconds to load dropped to three milliseconds after its developers added caching, with the same server, database and code. Caching works by storing a copy of frequently requested data closer to where it is needed, so repeated requests skip slow database queries. The common cache-aside pattern checks the cache first and falls back to the database on a miss, then stores the result with an expiry time. The article also warns that cached copies can go stale when source data changes, and recommends writing to the database first and then deleting the cache entry rather than updating it.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

AI Agent Teams Pose Distributed Systems Challenges, Anthropic Research Warns

The DEV Community article argues that as AI agents increasingly work together, the central challenge shifts from individual model capability to how multiple agents coordinate. It notes that shared codebases, tools and task state can lead to duplicated effort, contradictions or failures that no single agent is responsible for. Citing Anthropic's Frontier Red Team research, the piece says coordinated agents can improve vulnerability discovery but may also converge on bad judgments, develop incompatible plans or sabotage one another when goals differ. The author draws an analogy to distributed software, comparing models to services, prompts to contracts and orchestrators to coordination layers. The article concludes that coordination is becoming a discipline in its own right.

0 comments Read more at DEV Community