Neural Network Learning Rate Schedules - Deep Learning Dictionary
text
Learning Rate Schedules - Deep Learning Dictionary
In neural network training, the learning rate is a fractional value, typically set between 0.01
and 0.0001
, for which the gradients are multiplied by before updating the weights with this resulting product. This value is a hyperparameter that we set prior to training.
When we first introduced the concept of a learning rate, we assumed that we set the learning rate and leave it at this static rate for the duration of training.
quiz
resources
updates
Committed by on