Deep Learning Dictionary - Lightweight Crash Course

Neural Network Weights - Deep Learning Dictionary

An artificial neural network is made up of multiple processing units called nodes or neurons that are organized into layers. These layers are connected to each other via weights.

Each weight represents the strength of the connection between the two nodes it connects.

When the network receives an input at a given node in the input layer, this input is passed to nodes in the next layer via connections, and the input will be multiplied by the weight values assigned to those connections.

The weight values are first randomly initialized and then learned, updated, and optimized by the network during the training process.

For each node in a fully connected layer, a weighted sum is computed with each of the incoming weights. This weighted sum is considered the pre-activation output from the node.

This output will be passed to a non-linear activation function before passing the final output as input to the following layer via the connected weights.