Take-home task grades how LLM agents handle request-budget exhaustion

·1 views

A new take-home assignment for LLM engineers evaluates candidates on how their agent behaves when a request quota runs out, rather than on the success path. The exercise centers on a three-step worker that must persist progress and resume without repeating completed steps. A deterministic fault-injecting proxy returns 429 errors after a fixed number of calls, so every candidate faces the same failure point. Graders assess the terminal transcript and code, including distinct exit codes for success, quota exhaustion, and unexpected errors. The author argues quota exhaustion is common in production yet rarely tested, and that the task can be finished in about ninety minutes.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Why IBM's Granite name became shorthand for built-in AI guardrails

IBM's open Granite family of enterprise AI models has become a metaphor for guardrails, with the name meant to evoke durability and structural integrity. The article says Granite models ship with alignment layers, configurable output filters, and safety mechanisms built in during pretraining, rather than added afterward. It draws a comparison to granite's geological formation, describing safety as embedded in the material instead of bolted on. The metaphor reportedly appears in IBM documentation, technical briefings, and Brazilian regulatory discussions on AI governance. The piece also notes that the models support fine-tuning and retrieval-augmented generation while keeping core safety constraints intact.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Moving a Lovable Cloud App to Your Own Supabase Project: Step-by-Step

A new guide explains how to migrate an app built on Lovable Cloud to a Supabase project the owner controls. Because Lovable Cloud runs on a Supabase project managed by Lovable, it cannot be linked to a personal account, so the move requires creating a new project, copying the database and files, and switching the app's connection settings. The process covers exporting and restoring the database, which carries user accounts and password hashes, while sessions end and files, secrets, sign-in provider settings and connectors must be transferred manually. The guide advises keeping the old backend running until the new one passes all checks, and deleting nothing in Lovable until verified copies are held.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How Freelancers Can Deduct AI Subscriptions and API Costs at Tax Time

Self-employed people can generally deduct AI services they pay for as business expenses, though rules differ by country and a local tax authority or accountant should confirm the treatment. Subscriptions like ChatGPT are typically filed under internet or software services, with service fees also commonly accepted, and the same category should be used consistently across years. For accounts used for both work and personal purposes, only the business-use share should be deducted, based on a documented basis such as periodic review of chat history. Pay-as-you-go API charges usually fall under the same category, with foreign-currency costs converted at payment time. Costs like courses, books, and equipment have their own treatment, and avoiding a catch-all miscellaneous category keeps records clearer.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

agentgc queues heavy builds for parallel Claude Code agents via a hook

A developer describes running several Claude Code agents on one laptop, where simultaneous heavy jobs such as test runs and builds can overload the machine. They built agentgc, a CLI whose Claude Code hook rewrites heavy commands like cargo test or npm run build to run through a queue. By default, a quarter of the logical cores (at least one) run heavy work at once, and the rest wait for a free slot. The hook asks for approval before rewriting commands, with an --allow option to skip that prompt, and it only rewrites plain single commands, not those with shell syntax. The release notes say tests pass on Windows, Linux and macOS in GitHub Actions, but the full flow was only tested by hand on Windows, and unsigned binaries may trigger OS warnings.

0 comments Read more at DEV Community