Generate Image from Text

alpha2phi
2 min readMar 26, 2021

Text to image using Jupyter Notebook on Google Colab.

Overview

In one of my articles, I went through code examples on image classification using CLIP and ResNext. In this article let’s go through a few algorithms and libraries which can be used to generate an image from text.

CLIP + BigGAN

This library uses OpenAI’s CLIP and the generator from a BigGAN to generate images from text.

  • From Colab, open this notebook.
  • I use 100 iterations and 5 epochs to reduce the time to generate the image.

And this is the image generated for the phrase “flying cartoon elephant”.

Generated Image by Big Sleep

CLIP + Siren

This library uses OpenAI’s CLIP and Siren to generate images from text.

  • From Colab, open this notebook.
  • I use 100 iterations and 5 epochs to reduce the time to generate the image. Increase them for a better image.

And this is the image generated for the phrase “flying cartoon elephant”.

Generated Image using Deep Daze

DALL-E

OpenAI’s DALL-E is the new neural network that can be used to generate images from text. It is a huge 12-billion parameter version of GPT-3.

The DALLE-PyTorch library is an implementation of the paper. This model takes time to train but shows amazing results. A good explanation is available in this video.

  • From Colab, open this notebook.
  • I use the OpenAI model with a depth of 6 to reduce the time to generate the image.

And this is the image generated from the phrase “fireflies in a field under a full moon

Generated Image using DALL-E

References

If you are not a Medium member yet and want to become one, click here. (A part of your subscription fee will be used to support alpha2phi.)

alpha2phi

Software engineer, Data Science and ML practitioner.