Learning Under Distribution Shift: Sequential Detection and Online Adaptation

Machine learning has evolved from a predictive tool into a cornerstone of critical decision-making in science and industry. However, the reliability of such systems rests on the assumption that the data encountered during deployment will resemble the data seen during training. In dynamic, real-world environments, this assumption frequently breaks. The resulting phenomenon, known as distribution shift, often leads to performance degradation that passes unnoticed, as standard validation metrics are unavailable at inference time, thereby undermining predictive reliability.

In this talk, I present a methodological framework that unites sequential hypothesis testing with online self-supervised learning to address both the rigorous detection of distribution shifts and the dynamic adaptation of models to the changing environments.

The first part focuses on the rigorous detection of distribution shifts. I will introduce a statistical framework designed to monitor data streams in the absence of ground-truth labels, and test for the presence of distribution shifts. Importantly, I will show that this method offers theoretical guarantees, including anytime type-I error control and a bounded expected time of detection, resulting in an effective and reliable monitoring tool for real-time detection of distribution shifts.

The second part bridges the gap between detecting distribution shifts and dynamically adapting deployed models to the observed shifts. Concretely, I will propose an online self-supervised adaptation mechanism, grounded in (online) optimal transport principles, that leverages the evidence of shift to update the model on the fly. This results in a test-time training mechanism that promotes invariance to dynamically changing environments, continuously aligning the model with the evolving distribution of the data.

If time permits, I will also briefly outline our work on controlled feature selection, where we developed an anytime-valid testing framework for data efficient identification of important features.

Ph.D. student under the supervision of Prof. Yaniv Romano.