Developer Builds Exoplanet Detector From NASA Data, Learns Hard Lessons on Imbalanced ML

A developer built a machine learning model to detect exoplanets from NASA Kepler telescope light curve data, only to find their first model achieved perfect recall by lazily labeling nearly every sample as an exoplanet. The core problem was severe class imbalance — fewer than 1% of roughly 5,600 training samples were actual exoplanets, making standard accuracy metrics deeply misleading. Fixes like SMOTE oversampling and threshold tuning proved ineffective, while WeightedRandomSampler and Focal Loss delivered the most meaningful improvements. The developer ultimately switched to a 1D Residual Network architecture with preprocessing steps including median filtering and per-curve normalization. Stratified 5-fold cross-validation revealed an average F1 score of 0.65 with high variance across folds, underscoring how sparse positive examples make single-run metrics unreliable.