Reinforcement Learning - Developing Intelligent Agents

Deep Learning Course 6 of 7 - Level: Advanced

Reinforcement Learning Series Intro - Syllabus Overview

video

expand_more chevron_left

text

expand_more chevron_left

Reinforcement Learning series introduction

What's up, guys? Welcome to this series on reinforcement learning! We'll first start out by introducing the absolute basics to build a solid ground for us to run.

We'll then progress onto more advanced and sophisticated topics that integrate neural networks into reinforcement learning. We'll also be getting our hands dirty by implementing some super cool reinforcement learning projects in code! Without further ado, let's get to it!

What is reinforcement learning?

Reinforcement learning (RL) is an area of machine learning that focuses on how you, or how some thing, might act in an environment in order to maximize some given reward. Reinforcement learning algorithms study the behavior of subjects in such environments and learn to optimize that behavior.

A commonly referred to domain that can illustrate the the power of reinforcement learning is in game playing. I know you've had to come across all the posts and the news and the papers about games that are being played by AI, right?

Take AlphaGo, for example – DeepMind's artificially intelligent Go player that beat the world champion of Go. Even more recently, what's gotten a ton of hype is OpenAI's team of five neural networks, called OpenAI Five, which defeated a team of top professionals in the very complex video game, Dota 2. These human-defeating AIs are being run via reinforcement learning algorithms.

So, back to the description of reinforcement learning we mentioned earlier. We said that reinforcement learning focuses on how a given subject might take actions in an environment in order to maximize some reward. Using a game as an example of an environment, reinforcement learning is concerned with how the player of the game can take actions, like making a move in a certain direction, in order to maximize its reward, which in a game setting, might be points.

The plan

So, what's going to be the approach in this series? Well, if you're anything like me and have tried to jump head first right into reinforcement learning, you may have gotten overwhelmed by all the confusing lingo, unknown acronyms, and abundance of mathematical notation.

I mean, we've got things like Markov Decision Processes, Q-Learning, DQNs, Bellman Equations, Policy Gradients, and not to mention, a bunch of math everywhere that looks like this: Markov Decision Processes, Q-Learning, DQNs, Bellman Equations, Policy Gradients, and not to mention, a bunch of math everywhere that looks like this:

\begin{eqnarray*} v_{\ast }\left( s\right) &=&\max_{a\in \mathbf{A}\left( s\right) }q_{\pi_{\ast }}\left( s,a\right) \\ &=&\max_{a}E_{\pi _{\ast }}\left[ G_{t}\mid S_{t}=s,A_{t}=a\rule[-0.05in]{0in}{0.2in}\right] \\ &=&\max_{a}E_{\pi _{\ast }}\left[ R_{t+1}+\gamma G_{t+1}\mid S_{t}=s,A_{t}=a\rule[-0.05in]{0in}{0.2in}\right] \\ &=&\max_{a}E_{\pi _{\ast }}\left[ R_{t+1}+\gamma v_{\ast }\left(S_{t+1}\right) \mid S_{t}=s,A_{t}=a\rule[-0.05in]{0in}{0.2in}\right] \\ &=&\max_{a}\sum_{s^{\prime },r}p\left( s^{\prime },r\mid s,a\right) \left[r+\gamma v_{\ast }\left( s^{\prime }\right) \rule[-0.05in]{0in}{0.2in}\right] \end{eqnarray*}

Well, fear not! Lucky for us, we're going to tackle this beast of reinforcement learning together step-by-step in a super smooth and intuitive way. As we progress, you'll start to feel more comfortable and confident in your abilities to work with reinforcement learning. Adding this skill to your machine learning toolbox will prove to be immensely useful and valuable!

Through this series, we'll get exposure to and gain an understanding of the intuition, the math, and the coding involved with reinforcement learning. Some parts of the series will be purely intuitive, others are going to be more mathematically involved, and then others are going to be more programmatically involved. We'll first focus more on the intuition and some of the math, and then we'll dive into coding reinforcement learning programs.

Reinforcement learning series syllabus

Here is the syllabus for this course:

  • Part 1: Introduction to Reinforcement Learning
    • Section 1: Markov Decision Processes (MDPs)
      • Introduction to MDPs
      • Policies and value functions
      • Learning optimal policies and value functions
    • Section 2: Q-learning
      • Introduction to Q-learning with value iteration
      • Implementing an epsilon greedy strategy
    • Section 3: Code project - Implement Q-learning with pure Python to play a game
      • Environment set up and intro to OpenAI Gym
      • Write Q-learning algorithm and train agent to play game
      • Watch trained agent play game
  • Part 2: Deep Reinforcement Learning
    • Section 1: Deep Q-networks (DQNs)
      • Introduction to DQNs
      • Replay Memory Explained
      • Training a Deep Q-Network
      • TTraining a DQN With Fixed Q-Targets
    • Section 2: Code project - Implement deep Q-network with PyTorch
      • Deep Q-Network Code Project Intro
      • Build Deep Q-Network in Code
      • DQN Image Processing And Env Management
      • Deep Q-Network Training Code
      • Solving Cart and Pole With a DQN

If at any point you hit a speed-bump along any of the more technical parts, don't be discouraged! Take a breath, focus, ask questions, and I promise you'll be alright. This stuff takes effort, and we've got you! ❀️

quiz

expand_more chevron_left
deeplizard logo DEEPLIZARD Message notifications

Quiz Results

resources

expand_more chevron_left
Welcome to this series on reinforcement learning! We'll first start out by introducing the absolute basics to build a solid ground for us to run. We'll then progress onto more advanced and sophisticated topics that integrate artificial neural networks and deep learning into reinforcement learning. We'll also be getting our hands dirty by implementing some super cool reinforcement learning projects in code! Without further ado, let's get to it! Sources: Reinforcement Learning: An Introduction, Second Edition by Richard S. Sutton and Andrew G. Bartow http://incompleteideas.net/book/RLbook2020.pdf Playing Atari with Deep Reinforcement Learning by Deep Mind Technologies https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf πŸ•’πŸ¦Ž VIDEO SECTIONS πŸ¦ŽπŸ•’ 00:00 Welcome to DEEPLIZARD - Go to deeplizard.com for learning resources 00:30 Help deeplizard add video timestamps - See example in the description 05:21 Collective Intelligence and the DEEPLIZARD HIVEMIND πŸ’₯🦎 DEEPLIZARD COMMUNITY RESOURCES 🦎πŸ’₯ πŸ‘‹ Hey, we're Chris and Mandy, the creators of deeplizard! πŸ‘€ CHECK OUT OUR VLOG: πŸ”— https://youtube.com/deeplizardvlog πŸ’» DOWNLOAD ACCESS TO CODE FILES πŸ”— https://deeplizard.com/resources ✨ Support collective intelligence, join the deeplizard hivemind for exclusive rewards: πŸ”— https://deeplizard.com/hivemind 🧠 Use code DEEPLIZARD at checkout to receive 15% off your first Neurohacker order: πŸ”— https://neurohacker.com/shop?rfsn=6488344.d171c6 ❀️🦎 Special thanks to the following polymaths of the deeplizard hivemind: Tammy Mano Prime Ling Li πŸ‘€ Follow deeplizard: Our vlog: https://youtube.com/deeplizardvlog Facebook: https://facebook.com/deeplizard Instagram: https://instagram.com/deeplizard Twitter: https://twitter.com/deeplizard Patreon: https://patreon.com/deeplizard YouTube: https://youtube.com/deeplizard πŸŽ“ Deep Learning with deeplizard: Deep Learning Dictionary - https://deeplizard.com/course/ddcpailzrd Deep Learning Fundamentals - https://deeplizard.com/course/dlcpailzrd Learn TensorFlow - https://deeplizard.com/course/tfcpailzrd Learn PyTorch - https://deeplizard.com/course/ptcpailzrd Natural Language Processing - https://deeplizard.com/course/txtcpailzrd Reinforcement Learning - https://deeplizard.com/course/rlcpailzrd Generative Adversarial Networks - https://deeplizard.com/course/gacpailzrd πŸŽ“ Other Courses: DL Fundamentals Classic - https://deeplizard.com/learn/video/gZmobeGL0Yg Deep Learning Deployment - https://deeplizard.com/learn/video/SI1hVGvbbZ4 Data Science - https://deeplizard.com/learn/video/d11chG7Z-xk Trading - https://deeplizard.com/learn/video/ZpfCK_uHL9Y πŸ›’ Check out products deeplizard recommends on Amazon: πŸ”— https://amazon.com/shop/deeplizard πŸ“• Get a FREE 30-day Audible trial and 2 FREE audio books using deeplizard's link: πŸ”— https://amzn.to/2yoqWRn 🎡 deeplizard uses music by Kevin MacLeod πŸ”— https://youtube.com/channel/UCSZXFhRIx6b0dFX3xS8L1yQ πŸ”— http://incompetech.com/ ❀️ Please use the knowledge gained from deeplizard content for good, not evil.

updates

expand_more chevron_left
deeplizard logo DEEPLIZARD Message notifications

Update history for this page

Did you know you that deeplizard content is regularly updated and maintained?

  • Updated
  • Maintained

Spot something that needs to be updated? Don't hesitate to let us know. We'll fix it!


All relevant updates for the content on this page are listed below.