Deep Learning Fundamentals - Classic Edition

Deep learning and artificial neural networks for beginners

This series covers and explains concepts that are fundamental to deep learning and artificial neural networks for beginners.

In addition to covering these concepts, we also show how to implement some of the concepts in code using Keras, a neural network API written in Python. We will learn about layers in an artificial neural network, activation functions, backpropagation, convolutional neural networks (CNNs), data augmentation, transfer learning and much more!

We'll discuss different terms, understand what they mean, and how they fit in to the overall deep learning framework. In some posts, we'll also discuss how certain topics are implemented in code using Python and Keras.

We have a separate series that specifically focuses on how to use Keras, so be sure to check that one out as well. Many posts in the Keras series assume you already have a basic understanding of the deep learning topics that we'll be discussing here in this series, so keep that in mind.

If you want to follow along with the code implementation we'll do in this series, then at least check out the prerequisites video in the Keras series so you can get everything you need in order to make use of Keras going forward.

Syllabus

Below is the syllabus for parts 1 and 2 of the course. Check back for updates regarding the potential for part 3 to be added in the future.

• Part 1: Intro to deep learning
• Section 1: Artificial neural network basics
• Intro to machine learning
• Intro to deep Learning
• Neural network architecture
• Training neural networks
• Loss and learning rate
• Section 2: Data topics for deep learning
• Data sets for deep learning
• Predicting with neural networks
• Over and under fitting data
• Supervised, unsupervised, and semi-supervised learning
• Data augmentation
• One-hot encoding
• Part 2: Intermediate deep learning topics
• Section 1: Convolutional Neural Networks (CNNs)
• What are CNNs?
• Visualizing convolutional filters
• Zero padding
• Max pooling
• Section 2: Backpropagation
• Backprop intuition
• Mathematics of backprop
• Vanishing and exploding gradient
• Weight initialization
• Section 3: Additional deep learning concepts
• Bias terms in neural networks
• Learnable parameters
• Regularization
• Batches and batch size
• Fine-tuning and transfer learning
• Batch normalization

Now that we have an overview of the course, let's jump right into our first topic!

What is machine learning?

In this introduction, I'd like to start with the very basics.

We will ultimately be discussing deep learning, so we need to find out what that even means. To understand deep learning, we must first understand machine learning.

If we think about it, this almost sounds like what we would do with regular code:

1. Write an algorithm.
2. The machine executes the algorithm on a particular dataset.
3. Later, the machine can do the same task with data it has never seen before.

Okay. So how does machine learning contrast from just writing a logical algorithm to instruct a computer on how to do something?

Well, from the definition of machine learning that we just gave, the focus is on the “learn from that data” part of the definition.

With machine learning, rather than manually writing code with a specific set of instructions to accomplish a specific task, the machine is trained using data and algorithms that give it the ability to perform the task without being explicitly being told how to do so.

So this may sound like magic, but we'll see in later posts exactly how this is achieved. Let's look a basic example of how this can be done using machine learning verses traditional programming.

Machine learning vs. Traditional programming

Example: Analyzing the sentiment of a popular media outlet and classifying that sentiment as positive or negative.

Traditional programming approach

The algorithm may first look for particular words associated with a negative or positive sentiment.

With conditional statements, the algorithm would classify articles as positive or negative based on the words that it knows are positive or negative.

// pseudocode
let positive = [
    "happy", 
    "thankful", 
    "amazing"
];

let negative = [
    "can't", 
    "won't", 
    "sorry", 
    "unfortunately"
];

These are arbitrarily chosen by the programmer. Once we have the list of positive and negative examples, one simple algorithm is to simply count up the occurrences of each type of word in a given article. Then, the article can be classified as positive or negative based on which word count is higher, the positive examples or the negative examples.

Machine learning approach

The algorithm analyzes given media data and learns the features that classify what a negative article looks like versus a positive article.

With what it has learned, the algorithm can then classify new articles as positive or negative. As a machine learning programmer in this case, you won't be explicitly specifying the words for the algorithm to recognize. Instead, the algorithm will “learn” that certain words are positive or negative based on labels given to each article it examines.

// pseudocode
let articles = [
    {
        label: "positive",
        data: "The lizard movie was great! I really liked..."
    },
    {
        label: "positive",
        data: "Awesome lizards! The color green is my fav..."
    },
    {
        label: "negative",
        data: "Total disaster! I never liked..."
    },
    {
        label: "negative",
        data: "Worst movie of all time!..."
    }
];

Next stop: Deep learning

Now that we know generally what machine learning is, the next topic that we want to cover is deep learning. Deep learning is a tool or technique that can be used to implement machine learning. We'll be covering deep learning in detail as our next topic so stay tuned for that. I hope you found this article helpful!