Eight Chinese AI Models Tested on Same Coding Task via Pi Agent, All Pass

·1 views

A developer ran a controlled benchmark pitting eight Chinese AI models — including GLM-5.3, Kimi K3, Qwen3, DeepSeek-V4, and MiniMax-M3 — against an identical JavaScript coding task using the Pi coding agent. All eight models completed the task successfully across 45 agent requests, consuming 94,502 tokens at a total audited cost of just under four cents. The test was deliberately narrow, measuring only task completion, token usage, request count, and billing accuracy rather than broader coding ability. Each model ran in an isolated environment with the same configuration and task contract to ensure comparability. The author, who operates the API provider used in the runs, stressed that the results reflect operational metrics for this specific task and should not be interpreted as a general model ranking.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Developer cuts LLM inference costs 45% using self-hosted vLLM on cloud GPUs

A developer building autonomous AI agents and real-time LLM-driven applications found commercial API costs unsustainable due to high volumes of tool calls and token usage. To address this, they deployed a self-hosted inference cluster on cloud GPU platforms RunPod and Vast.ai using the open-source vLLM framework. The setup leveraged PagedAttention, continuous batching, speculative decoding, and prefix prompt caching to optimize performance and memory usage. Benchmarks across 1,000 synthetic agentic prompts showed average Time-To-First-Token dropped to 172ms, compared to roughly 420ms for commercial APIs. The configuration also reduced inference costs by approximately 45% to 74%, bringing estimated spend per million requests down from $1,850 to around $480.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

RAG System Silently Indexed Only 62% of Corpus Due to NFS Blind Spot

A developer discovered that their self-hosted RAG knowledge base had only indexed 62% of its intended corpus, despite all monitoring dashboards showing healthy green statuses. The root cause was the indexer's reliance on inotify filesystem events, which do not propagate across NFS mounts, meaning documents written from remote machines were never detected. Because the system always returns an answer from whatever it has indexed, the coverage gap went unnoticed — the failure was in ingestion, not in the language model itself. The developer also highlighted a broader measurement pitfall: reporting only documents successfully indexed without defining the total eligible corpus turns a metric into a misleading figure. The fix involved replacing event-driven watching with periodic polling using stat(), which works across network filesystems by actively querying rather than waiting for notifications.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Ari Vale Joins DEV Community to Share Insights on Tools and Automation

Ari Vale, a self-described digital builder, has joined the DEV Community platform. Vale expressed enthusiasm about becoming part of the community. Their stated interests include practical tools, automation, and learning in public. The introduction post marks Vale's entry into the developer-focused online community.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Four Books That Reframe AI as a Social and Political Problem

A developer writing for DEV Community argues that understanding AI requires moving beyond technical literature toward social and philosophical texts. After reading Yuval Noah Harari's Sapiens, the author highlights how AI systems encode societal myths about human worth into algorithmic decisions rather than delivering neutral logic. Cathy O'Neil's Weapons of Math Destruction is cited for exposing how opaque algorithms structurally amplify inequality while shielding decision-makers from accountability. Bruno Latour's We Have Never Been Modern is described as the most challenging read, arguing that technology and values cannot be separated and that AI systems must be governed as social-technical hybrids. The author's central thesis, shaped by Blake's NEXUS, is that debates about AI safety miss the point unless they address who controls these systems and for what purpose.

0 comments Read more at DEV Community