Consultant Benchmarks DeepSeek, Qwen, Kimi and GLM on Cost, Speed and Quality

A developer and consultant spent six weeks benchmarking four leading Chinese large language models — DeepSeek, Qwen, Kimi, and GLM — while building an inference layer for a client project. Testing used 200 production prompts spanning coding, summarisation, Chinese-language Q&A, and creative writing, with metrics covering latency, throughput, and cost per million output tokens. DeepSeek's V4 Flash emerged as a standout for price-performance, delivering 58 tokens per second and an 89% HumanEval coding pass rate at just $0.25 per million output tokens. Kimi was the most expensive option, priced around $3.00–$3.50 per million output tokens across its entire catalogue, roughly 12 times costlier than the cheapest GLM and Qwen models. The author cautioned that with a sample size of 200 prompts per model, results indicate trends rather than definitive rankings, and noted DeepSeek's limited vision support as a practical drawback.