Running Stable Diffusion with an Old GPU

Diffusion-based AI art is one of the hottest topics of 2022, and one of the most earth-shattering announcements of the past month has been the August 2022 open-sourcing of one of these SOTA (state-of-the-art) models known as Stable Diffusion. That’s right, mere months after Open AI releases their big potential cash cow DALLE-2 model to the world, a new largely privately funded org based out of the UK called Stability.AI open sources their model to the world out of nowhere. This was especially shocking as research groups at Google working on similar (likely superior systems) such as Imagen and Parti only released their whitepapers in May and June respectively, holding off on any public release of models due to the potential for harm (i.e.: generating explicit or politically sensitive content).

Under the hood, (and similar to DALLE-2), Stable Diffusion is built on top of the idea of diffusion - a class of algorithms that has largely displaced GANs in recent years for image generation. The model, in a nutshell, learns how to denoise an image of random noise conditioned on some sort of data. This data can be text prompts, but it can also be other images. This effectively enables novices to become “award winning digital artists” with *ahem* minimal effort.

As of 2022, Stable Diffusion joins Midjourney, DALLE-2, and Craiyon/DALLE-mini as an AI text to image generator you can use right now and is the only one for which content filters can be entirely disabled.

Running Stable Diffusion (Locally)

Note: Last updated 9/10/2022.

You can access a version of Stable Diffusion online via Hugging Face.

However, for the “real” enthusiasts who are interested in digging into the models, there’s no substitute for running the open sourced model on your own machine. There are currently many forks of the original github release, but the most feature rich repo, with a fully fledged GradioUI (with image editing and prompt matrix support) is the stable-diffusion-web-ui repo.

  1. Clone the stable-diffusion-web-ui repo (highly likely to be subject to change) and follow the install instructions for your platform. Installation options for systems with even very low VRAM should be available.
  2. Get the checkpoints here and copy into models/ldm/stable-diffusion-v1
  3. Build docker image.
$ cp .env_docker.example .env_docker
# Edit the .env_docker file
$ docker compose up --build
  1. Open http://localhost:7860 in your browser

Here’s some pictures of “Robot holding a tacorito bowl inside the NYSE, stock photo” (check out that gettyImages watermark!)

Tacorito robot

comments powered by Disqus

Related Posts

Stable Diffusion - De-painting with Stable Diffusion img2img

Stable diffusion has been making huge waves recently in the AI and art communities (if you don’t know what that is feel free to check out this earlier post).

Read more

Making a Hugo Website The Full Stack Way pt 3 - Basic Infrastructure as Code (IaC) with Terraform

In the previous tutorial, we deployed a Google Cloud Storage bucket my manually click a bunch of items in the Google Cloud website to create resources in the cloud.

Read more

Making a Hugo Website The Full Stack Way pt 4 - CI/CD with Github Actions and Terraform

In the previous tutorial in our multi-part series on making a static website, we deployed a Google Cloud Storage bucket using Terraform.

Read more