Deep Learning Dictionary - Lightweight Crash Course

Kaiming (He) Weight Initialization - Deep Learning Dictionary

Before training a network, we can initialize our weights from a number of different weight initialization techniques.

As we've previously learned, the exact way in which the weights are initialized can impact the training process. Certain weight initialization techniques can cause instabilities during training, and others can be used to prevent instabilities.