Developer replaces 7B LLM with tiny Go classifier for email triage, cuts GPU use

A developer building an inbox triage agent replaced a 7-billion-parameter Qwen 2.5 model with a lightweight, CPU-based text classifier written in Go, dramatically reducing compute costs. The new system uses three ordered layers: deterministic rules for obvious cases, a small TF-IDF and logistic regression model for the rest, and an LLM only as a last resort for uncertain cases. The small classifier was trained in Python using scikit-learn on nearly 5,800 labelled emails across six categories, then exported as a JSON file for use in Go. The result is sub-millisecond inference with no GPU required, and cloud LLM calls have become rare. The author argues that many production AI tasks are fundamentally classification problems that do not require large language models at all.