Deep neural networks relax the need for manual feature engineering by learning how to generate discriminative features in an end-to-end blind training procedure. However, robust generalization of deep neural networks on unseen data is still a primary challenge when domain shift exists between the training and the testing data. In this talk, we explore using prototypical distributions to enable a model to autonomously update itself to incorporate observed distributional shifts. A prototypical distribution is a multimodal distribution that is learned by a model to encode the input distribution. Our idea is to use this property to discover distributional drifts in the input data. Through applications, we demonstrate that this idea can be used to improve existing algorithms in the area of domain adaptation, presentation attack detection, and image segmentation.