OpenAI winds down AI image generator that blew minds and forged friendships in 2022

DALL-E 2 couldn’t render images perfectly, but the imperfections gave the images a dreamlike, painterly quality that the testers, many of them already visual artists themselves, found attractive. It also began to impact their artwork in the physical world. “[It] challenged me to change the way I paint on canvas,” said Lapine.

During those early months after the launch of DALL-E 2, the group of testers remained close on Twitter, pushing the limits of an entirely new medium together. Lapine, then a resident of tech hub San Francisco, said she met two of her fellow DALL-E 2 testers in person at one point.

As the testers explored the intersection between language and visual arts, they began to think of themselves as explorers of “latent space,” which is a term for the compressed multi-dimensional neural network representation of everything the AI model has absorbed. (Later in 2022, Replit CEO Amjad Masad coined the term “latent space astronauts” on Twitter.) Each word in the prompt, like “a teddy bear on a skateboard in Times Square,” functioned like part of an address pointing the generator to a different spot in that conceptual latent space, bringing unique mixtures of visual elements to the fore.

“It was literally mind-blowing,” said inventor Danielle Fong in an interview with Ars Technica. Fong, a physicist who likes to merge science and art, was among the early DALL-E 2 testers. “I could feel my brain expanding as I began to explore the latent visual space encoded into the AI and started to make conceptual art at a rapid rate to illustrate my ideas and feelings. It was incredible to dive in and to have this aid to convert sometimes intense emotions into art.”

Baskin began to conceptualize parts of latent space as physical places and called them “zones,” meaning non-existent virtual worlds she could visit. “I’d find zones that would evoke creative ideas, and I’d follow curiosity to other rabbit holes,” she said. And the other testers would get involved, too. “We’d share worlds and try to find each others’ worlds, or we’d do narrative storytelling. We made up games.”

Bonneville also recalled the allure of exploring latent space though OpenAI’s tool. “Initially, the latent space felt like a consciousness-expanding place to spend time. Every new discovery of the weird things DALL-E could create felt like progress,” he said.

Eventually, all that exploration began to add up in terms of images generated, occupying digital real estate on each tester’s device. “I saved around 10,000 images on my computer and never used them ‘for anything’ or published them, and I had a beautiful time,” said Baskin. “The tool itself was the experience. It was beautiful and inspiring entertainment on its own.”

“So many ethical questions”

While the early DALL-E 2 testers experienced amazement and exhilaration, some outside that initial testing group expressed concern about the technology when people realized that DALL-E 2 gained its ability by absorbing the skills of human artists and photographers. To generate art, OpenAI trained DALL-E 2 by analyzing hundreds of millions of images scraped off the Internet without consulting rights holders, along with images licensed from Shutterstock without the knowledge of their creators. (“The data we licensed from Shutterstock was critical to the training of DALL-E,” said OpenAI CEO Sam Altman in October 2022.)

“I’ve never felt so conflicted using an emerging technology as DALL-E 2, which feels like borderline magic in what it’s capable of conjuring but raises so many ethical questions, it’s hard to keep track of them all,” wrote blogger Andy Baio in August 2022 just after gaining access to DALL-E. At that time, Baio identified three major questions about the “ethics of laundering human creativity,” as he put it in his piece:

• “Is it ethical to train an AI on a huge corpus of copyrighted creative work, without permission or attribution?”
• “Is it ethical to allow people to generate new work in the styles of the photographers, illustrators, and designers without compensating them?”
• “Is it ethical to charge money for that service, built on the work of others?”

As awareness of this ethical debate grew online, how people answered these questions proved immediately polarizing. Many users on social media (which tends to highlight the most polarizing views by nature of its design) threw themselves into two extreme camps: “AI art is amazing” or “AI art is a travesty.”

Caught in between were moderates like Baio, who were amazed by the technology but still felt there were some important unresolved questions about ethics and the impact it might have on artists. Dr. Margaret Mitchell, who studies AI ethics at Hugging Face felt similarly. “I was impressed by how far we’d come technologically but wary about where the data had come from,” she said.

At first, OpenAI’s tight lid on things kept the controversy to a minimum. It claimed ownership of all generations, included a built-in content filter for violence, famous people, and sex, and added a small corner watermark to each generated image. And only roughly 200 people on earth could make images with OpenAI’s level of detail. In that small-bubble ecosystem—before DALL-E 2 went wide release in September 2022—and with the tech being so novel, the data scraping issues didn’t seem like such a problem at first. Nor did issues that emerged with bias and stereotypes in the dataset (though that topic did receive ample and appropriate criticism elsewhere thanks to researchers like Abeba Birhane).

But OpenAI wasn’t alone in pursuing image synthesis technology. AI imaging techniques published openly by machine learning researchers, such as CLIP and latent diffusion, had already allowed others to begin developing similar technology as well. Soon, users of other AI image models like Midjourney (which started as a closed beta in the spring of 2022) and Stability AI’s Stable Diffusion (launched fully in August 2022) began publicly cataloging those tools’ abilities to replicate the styles of living artists, often using artists’ names in guides to improve prompting techniques.

This wasn’t as much of an issue with DALL-E because OpenAI made an attempt to obscure the connection between certain types of metadata (like famous living artists’ names) and the images themselves, making it difficult to clone styles of existing artists. “In my experience, OpenAI’s model for DALL-E 2 didn’t have much knowledge of living artists’ work,” said Baskin. “In the beginning, we tended to enter special effects as prompts like ‘mirrored surface’ instead of artists’ names—that was more of the culture.”

But with the other image synthesis tools becoming publicly available in late 2022, the practice of referring to existing artists while conjuring AI-generated images in Midjourney and Stable Diffusion caused a backlash. On August 13, 2022, artist RJ Palmer started a viral thread on Twitter, writing, “What makes this AI different is that it’s explicitly trained on current working artists. You can see below that the AI-generated image (left) even tried to recreate the artist’s logo of the artist it ripped off. This thing wants our jobs, it’s actively anti-artist.”

The controversy over AI-generated art grew to a boiling point in the media, and we also covered it in detail at the time over dozens of articles. The issue later led to a lawsuit against Stability AI and Midjourney (but not OpenAI), along with ample protests in online artist communities that rejected AI image generators as exploitative and demeaning to working artists.

For the latent space astronauts, the glass dome that had initially contained their wonder and joy in exploring AI artwork—safely in the confines of OpenAI’s playground—had begun to crack.

Doubt, disillusionment, and fighting

Every early DALL-E 2 tester we’ve quoted here was an artist before the arrival of AI image synthesis, and each had to make their own ethical decisions about what AI art meant to them.

The DALL-E 2 testers told me that during the initial period before the public launch of Stable Diffusion (roughly April to August of 2022), they didn’t worry too much about the ethics of scraping artwork for training AI image synthesis models, but some doubts began to bubble up over time.

“DALL-E 2 felt special [compared to later models like Stable Diffusion] since it was still kinda wonky and didn’t know many art styles outside the obvious classics,” said Baskin. “A bigger question for me was: will this severely hurt artists if they’re already underpaid and people start using these images commercially?”

Lapine also said that during the early DALL-E 2 era, she was initially not too concerned about the ethics of training AI models on other people’s art and images scraped off the Internet. But that opinion soon changed when she searched through the image dataset that powered Stable Diffusion and discovered something shocking. “After my own experience of finding my medical record photos in the LAION dataset and hearing concerns from artists, I no longer feel comfortable interacting with these platforms,” she said. “There still isn’t enough transparency.”

Some of the testers began to ponder the future of copyright in the face of unlimited computer-generated artwork. Currently, the US Copyright Office and court decisions both agree that purely AI-generated images cannot receive copyright protection. But at that time, no one knew how those decisions would turn out. “Intellectual property in the age of AI needs to be totally rethought,” said Danielle Fong. “But it’s all pretty disorganized, and various artists are getting a lot of competition from AI. On the other hand, artists are now more empowered than ever. It’s certainly a complex issue.”

Bonneville said he took a more permissive view of AI art and copyright from the start. “I had zero concern for the ethics at first,” he said. “I still feel that if someone puts any creative work on the Internet, it should be expected that it will be used by fair use principles.”

Not long after the controversy over Stable Diffusion erupted, Bonneville spent days in threads on Twitter battling critics of AI-generated artwork. “I pushed hard in some comments against the one artist animator dude [RJ Palmer] who blew the conversation wide,” he recalled. “I argued on various threads but realized soon after things got to fever pitch that it was useless.” In retrospect, Bonneville feels like the controversy was much ado about nothing. “With what is coming in AI,” he said, “none of this was a battle worth fighting.”

Baskin has hope that even with AI art, human artists will still be valued since the AI models tend to blend every style together into a hypersaturated median representation of a broad sample of human-made artworks. “Since all modern AI art has converged on kind of looking like a similar style, my optimistic speculation is that people are hiring way more human artists now,” he said. “People’s interest in art has piqued, and now it’s more clear how magical and special human-made things are.”

“Some people clearly got lost in the latent space”

Even with the controversies, Bonneville relishes his memories of that brief time period in 2022. “Nothing will replace those early days of giddy, almost naïve elation,” he said. “It was a very fun time, and I met some really cool people.”

But as with any high, the astronauts eventually had to come back to earth. “Some of the novelty began to wear off once the reality sank in that latent space, for all practical purposes on a per artist basis, is infinite,” Bonneville said. “I think this realization served to cool a little bit of the dopamine reward mechanism.”

In the larger world beyond that core group of friends, AI elation quickly gave way to both the aforementioned ethics issues and also to intense commercial AI hype as ChatGPT replaced AI image generators as the flashiest new generative AI technology in late November 2022. Meanwhile, Bonneville began to get frustrated with social media AI art circles where the most provocative users received the most attention. The endless onslaught of AI-created images began to lose its luster.

“There was a subset of people that I stopped following that seemed to have developed a strong psychological need to create endless images, and soon people were talking about insane numbers that I cannot comprehend, like making 50,000 or 100,000 or 300,000 AI [images],” Bonneville said. “Some people clearly got lost in the latent space.”

Changes to Twitter itself—especially with the transition to Elon Musk’s ownership in October 2022—further fractured the AI art community as many people left the platform. Lapine said she has now lost track of most of the people in the community from those early days.

Bonneville likes to note that the collapse of the cryptocurrency-linked NFT market, where some AI art makers (outside of the group we’ve interviewed) hoped to earn money, added an extra taint to the AI-generating art community online. “Just last week, someone I had followed for quite a while but not kept up with too much recently posted a short tweet saying that they were done creating AI images,” said Bonneville. “In other words, the utopia that they hoped to discover somehow through AI art never materialized.”

Sunset for DALL-E 2

“Every few years, a technology comes along that splits the world neatly into before and after,” wrote Casey Newton in his July 2022 overview of DALL-E 2 for The Verge. We’re now living in the “after” part of his comparison, and two years later, the development of AI image generators hasn’t ceased. Adobe, Meta, Google, Stability AI, Microsoft, and Midjourney all offer access to commercial AI image synthesis models. OpenAI has moved on not only with DALL-E 3 but to wowing people with a synthetic video model named Sora.

With the speed of progress in AI image generators, DALL-E 2 was superseded in capability long ago, but it still kept a core group of users who were enamored with its distinctive style and outpainting features. Just last week, Will Douglas Heaven of MIT Technology Review profiled an artistic project called Synthetic Memories that used DALL-E 2 to create images of important family events that never happened. “I’m really scared that OpenAI will close DALL-E 2 and we will have to use DALL-E 3,” Haven quoted project creator Pau Garcia as saying.

Right now, when users visit the URL that previously hosted the DALL-E 2 interface on the OpenAI website, the site reads, “We are no longer allowing new users to DALL-E 2. DALL-E 3 has higher quality images, improved prompt adherence, and we’ve started rolling out image editing.” According to a spokesperson from OpenAI who spoke to Ars Technica, DALL-E 2 has turned off free credits and stopped selling credits to anyone who never bought them previously, though people with existing credits will be able to use them until May 1, 2025, or one year from purchase—whichever comes first. For now, the DALL-E 2 API remains available for developers.

At the time that OpenAI began sunsetting DALL-E 2, it still offered a few features that DALL-E 3 currently does not, such as the web interface itself and the ability to upload a photo and modify it with inpainting (modifying parts of an image) or outpainting (expanding the borders of an image). Currently, DALL-E 3 allows inpainting, but only on images it has already generated itself.

At the twilight of DALL-E 2, Lapine says that even though she moved on from AI-generated art platforms long ago, she still looks back fondly at that period of time. “I feel like we were a part of history and have this wild shared experience of being entrusted with new technology,” she said. “I miss the excitement and novelty of it. I miss the vivid dreams.”