Model Degradation: Monitoring Human Activity Recognition with Classical Drift Detectors

Abstract

This work investigates the impact of data drift on machine learning models deployed for Human Activity Recognition (HAR) using the UCI Human Activity Recognition Using Smartphones dataset. A Random Forest classifier is trained on an initial stable data distribution using a chronological split to reflect real-world deployment conditions. Model performance is first evaluated under no-drift conditions to establish a baseline. To study degradation, data drift is simulated by injecting Gaussian noise into the sensor features of future data, representing realistic changes such as sensor noise and calibration shifts. This controlled perturbation leads to a measurable decline in classification accuracy, precision, and recall, demonstrating the vulnerability of static models to evolving data distributions. Model degradation is monitored using the ADWIN (Adaptive Windowing) drift detection algorithm applied to the prediction error stream, enabling automatic identification of statistically significant performance changes. To mitigate degradation, a sliding-window retraining strategy is applied using recent samples, resulting in partial recovery of model performance. The results highlight the importance of time-aware evaluation, drift detection, and lightweight retraining for maintaining reliable machine learning systems in dynamic and resource-constrained environments.