Neural Network Programming - Deep Learning with PyTorch

How to Debug PyTorch Source Code

Welcome to deeplizard. My name's Chris. In this episode, we're going to see how we can debug PyTorch source code using the Visual Studio Code IDE.

Without further ado, let's get started.

Getting Started Debugging PyTorch

The first step, of course, is to get Visual Studio Code installed on your system. Once this is complete, we are ready to go.

A Program to Debug

To debug, we need a program that we can use to kick things off.

We have created a sample program here that we're going use to debug some PyTorch source code. We're going debug the Fashion MNIST dataset which actually lives in the torchvision package. But nonetheless, everything is going be the same.

import torch
import torchvision
import torchvision.transforms as transforms

train_set = torchvision.datasets.FashionMNIST(
    root='./data'
    ,train=True
    ,download=True
    ,transform=transforms.Compose([
        transforms.ToTensor()
    ])
)

image, label = train_set[0]

print(image.shape)

How to Debug

In order to debug any python code in Visual Studio Code, we need toe install the python debugging extension. This will give us debugging capabilities. Once the debugging extension is installed, we follow these steps.

1. Place a breakpoint
2. Run the program in debug mode
3. Use Keyboard to manually control program execution
4. Step into something PyTorch

Note that there is an additional requirement to debug imported libraries like PyTorch. We need to set the justMyCode attribute to false inside the launch.json file for VS Code.

Below is an example launch configuration.

{
    "name": "Python: Current File",
    "type": "python",
    "request": "launch",
    "program": "${file}",
    "justMyCode": false,
    "cwd": "${fileDirname}"
}

Debugging code is one of the absolute best ways to understand what's going on. Note that we don't have to write a lot of code to actually get set up to debug PyTorch. In our example it was just 15 or 16 lines of code.

To debug PyTorch source code, we simply need a path into the PyTorch code we want to debug.