LLMeter Lets Teams Track LLM Costs in Prometheus and Grafana Without a Proxy

·1 views

A tool called LLMeter enables engineering teams to monitor large language model (LLM) API costs directly within their existing Prometheus and Grafana observability stacks. Unlike most cost-tracking solutions, LLMeter does not require routing traffic through a third-party proxy, avoiding added latency and potential points of failure. It exposes a /metrics endpoint that Prometheus can scrape by adding a simple job entry to the prometheus.yml configuration file. This allows teams to visualize LLM spending in real time alongside infrastructure metrics such as CPU and memory usage. The approach aims to help developers correlate AI feature usage with billing spikes and treat LLM costs as a standard part of infrastructure monitoring.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Solon Framework Auto-Tunes Thread Pools at Runtime Using CPU Core Count

Solon, a Java application framework, sets its HTTP server thread-pool properties to zero by default, meaning the system automatically calculates optimal values based on the host machine's actual CPU core count at runtime. The auto-tuner uses a single key flag, server.http.ioBound, to determine whether a workload is IO-bound or CPU-bound, then applies different scaling formulas accordingly. For IO-bound services, maxThreads is set to 32 times the core thread count, while CPU-bound services use a leaner multiplier of 8 to avoid unnecessary context-switching overhead. This approach eliminates the common problem of hard-coded or copy-pasted thread-pool values that may have been tuned for a different server environment. Developers can still override individual settings in app.yml, but the zero-config defaults are designed to remain suitable for most deployments without manual intervention.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

GitPython CVE-2026-67324: One-character shorthand bypasses remote code execution guard

A critical vulnerability in GitPython, tracked as CVE-2026-67324 and scored 9.8 on CVSS 3.1, allows attackers to bypass the library's built-in guard against dangerous git options. The flaw affects version 3.1.50, where the security check blocks the long-form flag --upload-pack but fails to catch its short-form equivalent -u, enabling arbitrary command execution even when allow_unsafe_options is set to False. Exploitation requires that user-influenced input reaches the multi_options parameter of Repo.clone_from, a scenario common in CI pipelines, web forms, and tools that accept repository URLs or build parameters. GitPython is downloaded roughly 254 million times monthly on PyPI, arriving as a dependency of widely used tools such as MLflow, DVC, and Semgrep, significantly broadening the attack surface. Version 3.1.51 patches the issue, and this is reportedly the third bypass of the same security barrier within a single year, all sharing the same root cause.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Alibaba Launches Qwen3.8-Max, a 2.4T-Parameter AI to Rival OpenAI and Anthropic

Alibaba on Monday unveiled Qwen3.8-Max, its largest artificial intelligence model to date, targeting enterprise use cases such as software engineering and complex reasoning. The model uses a mixture-of-experts architecture with 2.4 trillion total parameters, though only around 95 billion are activated per inference cycle, balancing performance with speed and cost efficiency. Open-weight versions are set to be released publicly via Alibaba's cloud studio platform the following week. Internal benchmarks shared by the company suggest Qwen3.8-Max performs competitively against leading models from OpenAI and Anthropic on coding evaluations such as SWE-bench Pro. Alibaba also claimed the model autonomously completed three coding projects without human input, with one project running continuously for 16 days from an empty folder to a finished software product.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

AI Lets Anyone Build Apps, But Skipping Human Review Still Costs Dearly

A growing trend celebrates solo developers building apps over a weekend using AI tools, with no engineering team or formal background. However, critics argue that the absence of structured review processes — code reviews, QA testing, and managerial sign-off — leads to silent but consequential failures. Historical examples reinforce this: JPMorgan Chase's 2012 London Whale trading loss, partly traced to an unchecked spreadsheet formula error, cost the bank $6.2 billion. Similarly, a widely cited 2010 economic paper by Reinhart and Rogoff shaped years of government austerity policy before a graduate student discovered a formula error in 2013 that significantly altered its conclusions. The core argument is that review failures, not tool failures, are the recurring culprit — and AI-assisted development does not change that fundamental human blind-spot problem.

0 comments Read more at DEV Community