Dev Talk on AI

@SEADEV

Talk Notes

text

expand_more chevron_left

Introduction to Artificial Intelligence

What does the term artificial intelligence mean, and what is the difference between the following terms:

We can use the following diagram to motivate the differences.

Artificial Intelligence is a type of Problem Finding   consists of Problem Solving      consists of Machine Learning Deep Learning Parameters Humans   create   things that that are   have Creativity Computation Symbolic AI Expert Inventor Innovator Software Data Code Layered

Artificial Neural Networks

Let's look at a typical diagram of a basic neural network:

Input Hidden Hidden Hidden Input Hidden Hidden Hidden Output Output

Layers

This is a basic network that has \( 4 \) layers.

  • \( 1 \) Input layer (Green)
  • \( 2 \) Hidden layers (Black)
  • \( 1 \) Output layer (Red)

Data flows through this network from left to right.

Neurons (Nodes)

Each node in the network represents a number. The number of nodes in the input layer depends on the number of features in each sample of data. The number of nodes in the output layer depends on the number of labels or classes we have. The number of nodes in the hidden layers is arbitrary.

  • Input layer nodes (Data)
  • Hidden layer nodes (Arbitrary)
  • Output layer nodes (Labels)

Connections (Weights)

Each edge represents a parameter in the network. These parameters are called the network's weights. These values determine the network's mapping from input to output, and these are the values that are learned during the training process.

  • Define the mapping
  • Initialized randomly
  • Updated during training

Neural Network workflow

When we are working with neural networks, there are \( 3 \) main steps.

  • Prepare the data
  • Train the network
  • Predict with the network

Data

Suppose we have a dataset of people. For each person, we have \( 2 \) attributes, height and weight and \( 2 \) labels, \( 0 \) for female and \( 1 \) for male.

Sample Weight Height Label
\( 1 \) \( 50 \) \( 6 \) \( 0 \)
\( 2 \) \( 60 \) \( 7 \) \( 1 \)
\( 3 \) \( 75 \) \( 7.5 \) \( 1 \)
\( \vdots \) \( \vdots \) \( \vdots \) \( \vdots \)

Training

The goal of training is to update the weights so that the network gives accurate predictions. To do this, we pass the data through the network. This is called a forward pass. We use linear algebra to do this.

Then, we check the accuracy of the network, and update the weights so that the accuracy goes up. We update the weights by going backwards though the network. This is called a backward pass. We use calculus to do this.

Then, we repeat.

Here is the more verbose version of this process:

  1. Get batch from the training set.
  2. Pass batch to network.
  3. Calculate the loss (difference between the predicted values and the true values).
  4. Calculate the gradient of the loss function w.r.t the network's weights.
  5. Update the weights using the gradients to reduce the loss.
  6. Repeat steps 1-5 until one epoch is completed.
  7. Repeat steps 1-6 for as many epochs required to reach the minimum loss.

Prediction

Save the learned weights and deploy.

Neural Networks in Practice

Examples:

Developers and Deep Learning

In the past, developers would build applications primarily using text and numbers as input and output.

  • Text
  • Numbers

Deep learning enables developers to build applications that use images, audio, video and language as input and output.

  • Images
  • Audio
  • Video
  • Language

Deep Learning Libraries

  • PyTorch for Python
  • TensorFlow for Python
  • TensorFlow.js for Javascript