Neural Network Learning Rate Schedules - Deep Learning Dictionary
Learning Rate Schedules - Deep Learning Dictionary
In neural network training, the learning rate is a fractional value, typically set between
0.0001, for which the gradients are multiplied by before updating the weights with this resulting product. This value is a hyperparameter that we set prior to training.
When we first introduced the concept of a learning rate, we assumed that we set the learning rate and leave it at this static rate for the duration of training.
Committed by on