Embedding-Based Intent Detector Hits 92% Recall but Struggles With Negation and False Positives

A developer built a lightweight two-stage message classifier to detect actionable chat messages without calling a large language model on every input. The system uses sentence embeddings and cosine similarity, comparing each message against tool descriptions and neutral anchor phrases rather than relying on a fixed similarity threshold. Tested on 300 generated messages, the detector achieved 92% recall but only 76% specificity, meaning roughly one in four harmless messages incorrectly triggered an action badge. A key structural flaw was identified: embeddings cannot distinguish negation, so phrases like 'the outage is fixed, thanks' score nearly identically to 'outage.' The author argues this limitation is inherent to the embedding approach rather than a tuning problem, and recommends accepting the trade-off rather than attempting to engineer around it.