FreezeOut: Accelerate Training by Progressively Freezing Layers

06/15/2017
by   Andrew Brock, et al.
0

The early layers of a deep neural net have the fewest parameters, but take up the most computation. In this extended abstract, we propose to only train the hidden layers for a set portion of the training run, freezing them out one-by-one and excluding them from the backward pass. Through experiments on CIFAR, we empirically demonstrate that FreezeOut yields savings of up to 20 wall-clock time during training with 3 speedup without loss of accuracy for ResNets, and no improvement for VGG networks. Our code is publicly available at https://github.com/ajbrock/FreezeOut

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset