Fine-tuned AI model matches original on ranking but misses key job roles

A developer attempted to replace a costly AI job-filtering system with a fine-tuned local model. The fine-tuned model matched the original's ranking accuracy and significantly reduced false alarms. However, it failed to identify as many of the most promising 'strong' job roles, finding only 6 compared to the original's 10 out of 14. Due to this critical shortcoming, the original model remains in production, as missing a strong role is considered a more significant failure.