Does Double Descent Occur in Self-Supervised Learning?

07/15/2023
by   Alisia Lupidi, et al.
0

Most investigations into double descent have focused on supervised models while the few works studying self-supervised settings find a surprising lack of the phenomenon. These results imply that double descent may not exist in self-supervised models. We show this empirically using a standard and linear autoencoder, two previously unstudied settings. The test loss is found to have either a classical U-shape or to monotonically decrease instead of exhibiting a double-descent curve. We hope that further work on this will help elucidate the theoretical underpinnings of this phenomenon.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset