The Evolution of AI in the Realm of Artistry #

Before delving into the concept of Stable Diffusion, it is essential to explore the developmental journey of AI in the realm of painting.

As early as 2012, a team led by Chinese scientist Andrew Ng achieved a groundbreaking milestone in the field of deep learning. They trained the world’s largest deep learning network at that time, which had the remarkable ability to autonomously recognize objects like cats. Within a mere three days, this network generated a blurry yet discernible image of a cat. Despite its fuzziness, this image demonstrated the immense potential of deep learning in the domain of image recognition.

Unveiling the Journey of AI in the Realm of Artistry

In the year 2012, a team led by the visionary Chinese scientist, Andrew Ng, accomplished a remarkable feat by training the world’s largest deep learning network. This network demonstrated the power of deep learning by autonomously recognizing objects such as cats and producing a blurry yet identifiable cat image within a remarkably short span of three days. This image, albeit its lack of clarity, showcased the immense potential of deep learning in the realm of image recognition.

Fast forward to 2014, Ian Goodfellow, a Google scientist from the University of Montreal, introduced the algorithm of Generative Adversarial Networks (GANs). This algorithm quickly emerged as the dominant approach for AI-generated artwork. GANs operate on the principle of training two deep neural network models, namely the Generator and the Discriminator. The Generator learns to generate novel data samples that resemble real data, while the Discriminator accurately distinguishes between the generated fake samples and real data. The core idea behind GANs is a strategic game where the Generator aims to deceive the Discriminator while the Discriminator strives to discern truth from fiction. Through this adversarial and collaborative process, GANs ultimately achieve the generation of high-quality data.

In 2016, the first text-to-image model based on GANs, called GAN-INT-CLS, emerged, showcasing the feasibility of GANs in generating images from textual descriptions. This breakthrough opened the floodgates for various GAN-based conditional image generation models. However, GANs often encountered issues of instability and collapse during the training process, posing challenges for large-scale applications.

In the same year, October saw the introduction of ProgressiveGAN by NVIDIA. This technique gradually increased the scale of neural networks to generate high-resolution images, thereby mitigating training difficulties and enhancing the quality of generated output. This paved the way for the subsequent rise of StyleGAN.

In 2017, Google published the influential paper “Attention Is All You Need,” introducing the Transformer architecture, which subsequently revolutionized the field of natural language processing. Although initially designed to address language-related tasks, the Transformer displayed great potential in the domain of image generation. In 2020, they further proposed the concept of Vision Transformer (ViT), attempting to replace traditional Convolutional Neural Network (CNN) structures with the Transformer architecture in computer vision applications.

The year 2020 marked a turning point. UC Berkeley introduced the well-known Denoising Diffusion Probabilistic Model (DDPM), simplifying the loss function of existing models and shifting the training objective towards predicting the noise added at each step. This remarkably reduced training complexity, along with the replacement of network modules with the more expressive Unet architecture, improving the model’s representational capacity.

In January 2021, OpenAI unveiled two models: DALL-E, based on the VQVAE model, for text-to-image generation, and CLIP (Contrastive Language-Image Pre-Training) for learning the correspondence between text and images. These advancements seemed to offer AI a deeper understanding of human descriptions and creative processes, igniting an unprecedented enthusiasm for AI-generated artwork. In October 2021, Google released the Disco Diffusion model, which showcased astonishing capabilities in image generation, thus heralding the era of diffusion models.

In February 2022, the emergence of the Disco Diffusion-based AI drawing generator, developed by engineers from open-source communities, propelled AI art into a rapidly advancing trajectory. The Pandora’s box had been opened wide. Disco Diffusion proved to be more user-friendly than traditional AI models, with comprehensive documentation and supportive communities established by researchers. As a result, more and more people began to pay attention to its potential. In March of the same year, MidJourney, an AI generator developed by core contributors of Disco Diffusion, was officially launched. It found its home on the Discord platform, utilizing a chat-based human-machine interaction approach, simplifying operations by eliminating the need for complex parameter adjustments. A mere input of text in the chat window would yield awe-inspiring images.

Most notably, the images generated by MidJourney were so astonishingly impressive that they could hardly be distinguished by non-experts as creations of AI. After five months of MidJourney’s release, the results of an art competition held at the Colorado State Fair were unveiled, and to everyone’s surprise, a painting titled “Space Opera House” claimed the first place. However, this masterpiece was not the creation of a human artist, but rather the imaginative output of an artificial intelligence known as MidJourney.

The revelation that the winning artwork had been created by an AI sparked widespread anger and anxiety among human artists.

On April 10, 2022, OpenAI released DALL-E 2, which took the concept of AI-generated art to an entirely new level. While observant viewers could still discern the works produced by Disco Diffusion or MidJourney as AI-generated, the images generated by DALL-E 2 became indistinguishable from those created by human artists.

Stable Diffusion #

On July 29, 2022, the AI generator developed by Stability.AI called Stable Diffusion entered its beta testing phase. People quickly discovered that the quality of AI-generated artwork produced by Stable Diffusion rivaled that of DALL-E 2, with even fewer limitations. The beta testing of Stable Diffusion consisted of four waves, inviting 15,000 users to participate. Just ten days later, a staggering 17 million images had been generated through this platform. What’s more, Stability AI, the company behind Stable Diffusion, adhered to the ethos of openness, embracing the concept of “AI by the people, for the people.” This meant that anyone could deploy their own AI art generator locally, truly enabling every individual to “create a painting as long as you can speak.” The open-source community, HuggingFace, quickly adapted to this breakthrough, simplifying personal deployment. Meanwhile, the open-source tool, Stable-diffusion-webui, integrated various image generation tools, allowing users to fine-tune models and train their personalized models even on the web. Highly acclaimed, the tool garnered 34,000 stars on GitHub, marking the significant progress of diffusion model deployment from large-scale services to personal deployment.

In November 2022, Stable Diffusion 2.0 was released, offering four times the resolution of generated images and faster generation speeds.

Based on Latent Diffusion Models, Stable Diffusion placed the most time-consuming diffusion process in the low-dimensional latent space, significantly reducing computational requirements and the threshold for personal deployment. The factor of 8 used for encoding the latent space translates to reducing the width and height of the image to one-eighth of their original size. For example, a 512x512 image directly becomes 64x64 in the latent space, resulting in a saving of 64 times the memory! Moreover, Stable Diffusion lowered the performance requirements. It could rapidly generate a high-resolution 512x512 image with rich details, with only an 8GB Nvidia consumer-grade 2060 graphics card. Without this compressed latent space transformation, it would require a supercard with 512GB of memory