Developer builds post-work AI pricing system using a calibrated judge model

A developer experimented with a usage-based API pricing model where cost is determined after the work is completed, rather than by token or call count. They used a lightweight model called Jev to assess the complexity of each AI-generated answer and assign a price based on probability distributions across three work tiers. Instead of rounding probabilities into fixed buckets, the system charges the statistical mean, allowing prices to reflect nuanced output quality more accurately. A grounding check was also incorporated, multiplying the expected price by how well the answer was supported by actual source material. The project highlighted that calibrated probability distributions carry more pricing information than discrete tiers, and that verification checks require relevant context to produce meaningful results.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in