Generate Image from Text

Text to image using Jupyter Notebook on Google Colab.

Overview

In one of my articles, I went through code examples on image classification using CLIP and ResNext. In this article let’s go through a few algorithms and libraries which can be used to generate an image from text.

CLIP + BigGAN

This library uses OpenAI’s CLIP and the generator from a BigGAN to generate images from text.

  • From Colab, open this notebook.
  • I use 100 iterations and 5 epochs to reduce the time to generate the image.

And this is the image generated for the phrase “flying cartoon elephant”.

Generated Image by Big Sleep

CLIP + Siren

This library uses OpenAI’s CLIP and Siren to generate images from text.

  • From Colab, open this notebook.
  • I use 100 iterations and 5 epochs to reduce the time to generate the image. Increase them for a better image.

And this is the image generated for the phrase “flying cartoon elephant”.

Generated Image using Deep Daze

DALL-E

OpenAI’s DALL-E is the new neural network that can be used to generate images from text. It is a huge 12-billion parameter version of GPT-3.

The DALLE-PyTorch library is an implementation of the paper. This model takes time to train but shows amazing results. A good explanation is available in this video.

  • From Colab, open this notebook.
  • I use the OpenAI model with a depth of 6 to reduce the time to generate the image.

And this is the image generated from the phrase “fireflies in a field under a full moon

Generated Image using DALL-E

References

Programmer and occasional blogger.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store