Embedding Lookups Can Replace Full Taxonomy Prompts in LLM Classifiers

Classifying e-commerce search queries into product taxonomies is a common but costly LLM task, as large catalogs with hundreds of categories must be sent with every prompt. This token overhead forces teams to use larger, more expensive models, raising per-request costs significantly. A proposed alternative skips sending the taxonomy entirely: a small model generates plausible-sounding but fictional category labels for a given query, which are then matched to real taxonomy nodes via embedding similarity search. The approach exploits the fact that a model needs to understand a query's meaning, not memorize a category tree, to produce a useful classification signal. Developers are advised to measure accuracy and failure modes carefully before relying on this pattern in production.