AI Art for Beginners - Stable Diffusion Crash Course

Software Installation and Environment Setup

Let's talk about what we need in terms of resources for this course. Generating images using Stable Diffusion is the core of what we want to do. This is the goal. The core components that make this possible are as follows:

• GPU (Graphics Processing Unit)
• Git (Source Code Management)
• Miniconda (Python)
• Stable Diffusion Web UI (Automatic1111)
• Stable Diffusion Model (Artificial Intelligence)

Later in the course, we will need these as well:

• FFmpeg (Digital Media Processing)
• Davinci Resolve (Professional Editing Software)
• Course Resources Download

We recommend coming back to these items later when they are needed.

For now, the ultimate goal is to be able to generate an image using Stable Diffusion. We don't want to get slowed down installing software that won't be used until later in the course.

Let's begin by look at the GPU requirements.

GPU (Graphics Processing Unit)

It is possible to use a CPU (Central Processing Unit) to generate images with Stable Diffusion. However, this process will take a long time, possibly hours to generate a single image. A GPU speeds things up to only seconds.

There are two options in terms of following along and running the examples in this course:

• Local GPU
• Hosted GPU (Google Colab)

If you have a GPU on your computer (local), the best way to know if it will work with Stable Diffusion is to test it. Many GPUs have been shown to work with Stable Diffusion. This is one of the innovations that Stable Diffusion gave us. The ability to run on consumer laptops. Go to the quick setup section to see how to test your system to see if it works.

If you don't have a local GPU or you don't won't to install software on your computer, Google Colab is an option that can be used.

It's important to note that Google Colab has restricted the use of the Automatic1111 UI under the free version. It's no longer possible at the moment to use the Automatic1111 UI with Google Colab for free.

The Colab product lead has said that the team cannot support the usage growth of the Automatic1111 UI and Stable Diffusion on their budget. This was mentioned in this tweet.

It is still possible to use the UI with a paid plan. However, the compute units may go fast. Keep this in mind and checkout the possibility of purchasing compute units with Google Colab plans. Also, go to the lesson on setting up Stable Diffuion & Automatic1111 with Google Colab & Google Drive to get up and running.

Quick Setup for Automatic1111 & Stable Diffusion

This is a quick overview for getting up and running with Stable Diffusion on a local GPU. See the specific sections below for more details.

1. Download and install Git
2. Download and install Miniconda
3. Search for the Miniconda app:
   • Select: Anaconda PowerShell Prompt (miniconda3)
   • Create an environment:
   • conda create -n sd python=3.10.6 -y
   • Activate the environment:
   • conda activate sd
   • Fix a bug related to versions:
   • pip install fastapi==0.90
4. Download the Stable Diffusion Web UI
   • From the Miniconda prompt, run:
   • git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git
   • Move into the downloaded folder:
   • cd ./stable-diffusion-webui
   • Set the version of the web app to the one used in this course:
   • git checkout 2c1bb46c7ad5b4536f6587d327a03f0ff7811c5d
   • Run the app, and it will install dependencies:
   • python ./launch.py
5. Download Stable Diffusion
   • HuggingFace download link: sd-v1-4.ckpt
   • Place the .ckpt file into web app models directory:
   • ./stable-diffusion-webui/models/Stable-diffusion
6. Test the Automatic1111 install:
   • In the Miniconda prompt, navigate to the web app folder:
   • ./stable-diffusion-webui/
   • Run the following command:
   • python ./launch.py
   • Browse to the following url:
   • http://127.0.0.1:7860
   • Add a prompt and click Generate.

Here are a couple of options to consider when running the web app:

# Use this if you are running on Linux
python ./launch.py --xformers

# Use this if you will be accessing the UI from a remote server
python ./launch.py --listen

# Use this if you have a small amount of GPU memory 
python ./launch.py --medvram

# Use this if you have an even smaller amount of GPU memory 
python ./launch.py --lowvram

# Use this if you are running on an AMD GPU
python ./launch.py --precision full --no-half

For AMD GPU help, see: Install and Run on AMD GPUs

For Mac help, see: Installation on Apple Silicon

Git (Source Code Management)

We recommend using git, but it is strictly necessary. Git makes it easy to pull code from GitHub. Git can be downloaded from the official site here: https://git-scm.com

Note that Git comes installed by default on most Mac and Linux machines. To check if you have Git installed. Open the Terminal app and type git --version and press enter.

Miniconda

To run Stable Diffusion, we need to have Python installed. To install Python, we recommend using Miniconda, a lightweight installer for Conda which is a Python environment manager.

Whenever we install Miniconda, we get Conda, and Conda makes it easy to get Python and all of the additional software packages associated with Python.

• Download Miniconda

Once Miniconda is downloaded, go through the installation process. After Miniconda is installed, we can open a Miniconda PowerShell prompt (Windows) or type conda in our terminal (Mac & Linux). Whenever conda is recognized at our prompt, we are ready.

Next, we need to create and Python environment for our work. At the prompt, we run the following commands:

conda create -n sd python=3.10.6 -y
conda activate sd

This creates an environment called sd, short for stable diffusion, and uses Python 3.9. The activate command specifies that we want to use the sd environment. To verify that Python is installed an ready, we type the following:

python --version
#output: Python 3.10.6

That's it! Our Python Stable Diffusion environment is ready. The Stable Diffusion Web UI comes with scripts that will automatically setup the rest of the environment.

Stable Diffusion Web UI (Automatic1111)

The next piece of software we need to download is the Stable Diffusion Web UI by GitHub user Automatic1111. This UI is a powerful open source tool for interacting with Stable Diffusion. We will sometimes hear the UI referred to as the Automatic1111 or Auto1111 interface.

If you are already a git user, you can install the interface using git. If you want to try using git for the first time, install it here. Otherwise, a copy of the Automatic1111 interface has been included as a .zip in the course resources allowing git to be bypassed.

• ./stable-diffusion-webui.zip

Unzip this folder to install the UI without using git.

To install the Automatic1111 interface using git, run the following command:

git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git

Then, if you want to be sure that your interface matches what's in the course, checkout the commit that was using for the course using the following command from the ./stable-diffusion-webui folder:

git checkout 2c1bb46c7ad5b4536f6587d327a03f0ff7811c5d

After installing the UI, we want to open our Miniconda prompt and navigate to the ./stable-diffusion-webui folder. Be sure that the sd environment is activated using conda activate sd, and we want to run the following:

python .\launch.py

Note that both git users and users that did not use git should do this.

This will launch the web UI. The first time the web UI launches, the UI will install all of the necessary software into the sd environment. This will take some time.

When then installs finish, the terminal tells us that we need a checkpoint to run.

Can't run without a checkpoint. Find and place a .ckpt file into any of those locations. The program will exit.

The .ckpt file is referring to the model. We need to download the model, and then, we will be able to run the UI. Let's get the model installed.

Stable Diffusion Model (Weights)

To download the Stable Diffusion Model, we need to use Hugging Face 🤗. Hugging Face is a popular website that hosts AI and Machine Learning resources.

When we say download the Stable Diffusion Model, we specifically mean the Weights of Stable Diffusion. Once we have Stable Diffusion's weights, we can create the model in code by loading in the weights.

We can think of the model's weights as its memory. The weights are updated during the training process. When the model learns during training, the information that is learned is stored in the weights.

Note that when we talk about a model being open source, we are referring to the model's weights in addition to the code required to run the model.

To download the weights, go to the Stable Diffusion page on the Hugging Face site click the sd-v1-4.ckpt link in the downloads section.

• Hugging Face weights file download link: sd-v1-4.ckpt

Note that the extension .ckpt is short for checkpoint. For this reason, we will hear this file referred to as a checkpoint file.

Another file extension that is on the rise is the .safetensors file. This type of file is becoming popular on Hugging Face because it has better security. For this reason, choose .safetenors files whenever possible.

Now, we want to move the sd-v1-4.ckpt file to the models directory of the Automatic1111 web app. Specifically, we want to put the file here:

./stable-diffusion-webui/models/Stable-diffusion

At this point, we are ready to run the web application again. This time the application should start and we can browse to the application and generate and image. This is the url:

• http://127.0.0.1:7860

If you run into the error below when running Automatic1111:

RuntimeError: Cannot add middleware after an application has started

Be sure the run the following command to resolve the issue: pip install fastapi==0.90

FFmpeg

In order to convert the images that we create into a video, we will be using FFmpeg, a popular command line tool for processing of video and audio files. The official site for ffmpeg can be found by following this link.

A copy of this application for Windows has been included inside the course resources folder. To get another version of ffmpeg or to download ffmpeg on Linux or Mac, go to the official downloads page.

To install on Windows, we create this file:

C:\Program Files\ffmpeg

Then, we manually place the ffmpeg files in this directory.

Finally, to access ffmpeg from the terminal, this path needs to be added:

C:\Program Files\ffmpeg\bin

To do this:

1. Search for Environment
2. Click on Edit system environment variables
3. Click on Environment variables...
4. Under System Variables, choose path and edit
5. Add the path: C:\Program Files\ffmpeg\bin

To run ffmpeg, we type ffmpeg in the terminal.

Note that we can run ffmpeg without adding to the path or putting the files in C:\program files by specifying a custom path to the ffmpeg.exe directly from the terminal.

Davinci Resolve

Davinci Resolve is a professional editing software application. It is packed with capabilities including:

• Video editing
• Color correction
• Visual effects
• Motion graphics
• Audio post production

Originally, Davinci Resolve was a color correction only system that was sold at a price of $250,000 to $850,000 depending on the configuration. Now, Resolve is packed with many more features, and we can use it for free.

There is a paid version that has additional features. However, the free version has everything needed for solo creators.

In the course, we will be using FFmpeg in our projects, but we will also be showing how an editing software like Resolve can be leveraged to further the creative possibilities.

To download Resolve, go to BlackMagicDesign website, and scroll down until you see the downloads button.

Course Resources Download

To get the course resources download, enroll in the course and navigate to the course page and click on the download option.

This download contains resource files like images, masks and scripts that we will be using throughout the course.

Practice with Flashcards

Challenge your grasp of the concepts by exploring these interactive flashcards.