Fine-tuned 310M encoder beats Jev on topic classification but ties on polarity tasks

A developer ran three Japanese text classification tasks — long-document topic, short-text polarity, and financial-sentence polarity — across six systems using 250 labeled rows each. A 310M ModernBERT encoder fine-tuned on just 250 labels outperformed TypeSafe's Jev decision API by 12 percentage points on topic classification, while also being 4 to 20 times faster. However, on the two polarity tasks, the trained encoder and Jev were statistically tied, with neither system holding a clear overall advantage. Zero-shot open-source models, including a GLiNER-family classifier, consistently underperformed Jev across all three tasks. The key finding is that which system wins depends on task structure — zero-shot APIs like Jev are practical when no labeled data exists, while small trained encoders gain an edge once a few hundred labels are available.