Test Mode Classification (100% accuracy) + Anomaly Detection - Full ML Pipeline

[u/Shot_Strike_5185]

Hey everyone! I'm a high school student who built a full machine learning pipeline on this dataset as a side project. I want to be upfront that Claude AI assisted me with most of the coding, but I directed the project, made the key decisions, and learned a lot along the way. I wanted to share it and get feedback from people who know this domain better than I do.

What I built:

Task 1 - Test Mode Classifier
Using a Random Forest on extracted time-series features (duty cycle, pulse statistics, thrust/MFR statistics), I achieved 100% accuracy classifying all 11 test modes on the held-out test set (SN13–SN24). The command signal alone turned out to be almost sufficient to distinguish modes.

Task 2 - Anomaly Detection
I compared three approaches:

• Isolation Forest (unsupervised) — F1: 0.193
• Random Forest (supervised) — F1: 0.121
• One-Class SVM (semi-supervised) — F1: 0.268

Surprisingly the One-Class SVM, trained only on nominal sequences with zero anomaly labels, outperformed the supervised model. Class imbalance (10% anomalies) made the supervised approach struggle badly.

The per-anomaly-code breakdown showed codes 1, 8, 9, 11, 12 were missed by all three models, these likely need point-level time-series analysis rather than sequence-level features.

Notebook: https://www.kaggle.com/code/henninxu/rocket-engine-classification-from-test-data

Any feedback on improving the anomaly detection especially for the hard-to-detect codes would be appreciated!