Developer creates tool to accurately measure AI token costs for self-hosted servers

·1 views

A developer who rents GPUs discovered that calculating the exact cost per AI-generated token is difficult due to fluctuating performance. Key variables like batch size, quantization, and GPU load affect the tokens-per-second rate, making single measurements unreliable. To address this, the developer built an open-source tool called Throttle that runs repeated tests to determine if configuration changes actually reduce costs. The tool provides clear results—CHEAPER, MORE EXPENSIVE, or NO WINNER—only when changes are statistically significant. It is available for installation via pipx and aims to bring clarity to cost optimization for self-hosted AI models.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Guide: Building a Custom CRM with Python and SQLite

A new tutorial details how to create a simple Customer Relationship Management system using Python and SQLite. The guide is aimed at entrepreneurs, developers, or small business owners seeking a cost-effective and customizable alternative to commercial CRM platforms. The project emphasizes data sovereignty, as information is stored locally instead of with external providers, and avoids recurring subscription fees. The step-by-step instructions cover setting up a development environment, designing a database schema, and building a functional application to track client interactions.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Colombia mandates electronic invoicing, guide outlines automation for SMEs

Colombia's DIAN has made electronic invoicing mandatory for most taxpayers since its 2022 resolution. The manual process poses an operational burden and error risk, especially for small and medium businesses. Automating the process can reduce time, eliminate human error, and ensure sustainable regulatory compliance. A practical guide explores the technical elements involved and how to implement an automated workflow using accessible tools and Python code. The process involves generating a specific XML format, applying a digital signature, and transmitting it to DIAN's validation system.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Guide: Add OIDC Authentication to Undertow Apps Using Pac4j Library

A technical guide explains how to secure a Java Undertow web application using the OpenID Connect protocol. Developers can integrate authentication by adding security, callback, and logout handlers from the undertow-pac4j library. The application can connect to external identity providers like Keycloak, Google, or Microsoft Entra ID. The example uses Java 17, Undertow version 2.4, and undertow-pac4j version 6.1.0.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Vaynerov's AI project estimator keeps pricing separate from language model

Vaynerov Technologies built an AI tool called Project Blueprint that generates project architecture and team plans from product descriptions. The system deliberately keeps pricing calculations separate from the language model, using an external calculation engine instead. All model suggestions for platforms and features undergo validation and sanitization before being sent to the pricing engine. This design ensures pricing remains consistent with business rules while making the AI's influence on recommendations transparent.

0 comments Read more at DEV Community