Feature Engineering Boosts One Class but Hurts Overall ML Competition Score

A machine learning practitioner found that adding manual signal features — Impact Severity and Energy Ratios — improved the F1 score for Class 2 but degraded performance for Classes 0 and 1, ultimately lowering the total leaderboard score. The root causes included multicollinearity from redundant features, dimensional mismatches from combining unscaled units, and extreme data spikes caused by division by small RMS values. To address these issues, fixes such as logarithmic transformation for outlier suppression and isolating noisy features into a binary sub-model were proposed. Multi-threshold post-processing using scipy's Nelder-Mead optimizer was recommended to tune prediction probabilities across all classes simultaneously. The exercise highlighted that strong leaderboard performance in ML competitions often depends more on post-processing strategies and distribution alignment than on domain-driven feature engineering alone.