Paint by algorithm: Can AI make art, or is it all just derivative?

At the Tate Modern in London last month, visitors walked through the exhibition rooms soaking in the short, multicolored brushstrokes with which French artist Paul Cezanne created his still lifes, portraits and landscapes and marveling at his creativity, talent and impact on 20th-century art.

But now, a century-plus after his death, artificial intelligence technology is able to replicate his work in seconds.

In fact, today’s AI systems can create any image in any style — from impressionism to cubism to pop art. All they need is a detailed prompt.

The creative world is abuzz with potential. Previous progress in AI made it possible for computers to compete with humans in analytical areas, leaving creative work to the artists, writers, and designers. Now, however, the new field of “generative AI” is giving machines the ability to create works that are completely new, drawing inspiration from the vast amount of online data and knowledge that has accumulated over centuries.

This has the potential to revolutionize human creativity, experts say, making professionals from software engineers to writers to artists dramatically reassess how they work.

The incursion of AI into the arts raises questions about how important human input will continue to be in the creative process. Can there be art without an artist? If the art is created by a machine, to whom does it belong? What are the hidden dangers to society and humanity? The Times of Israel spoke to experts to try and shed light on some of these issues.

The overall conclusion is that the creative industry has no choice but to embrace AI. Rather than ousting human artists, the technology will collaborate with them to create new kinds of works, further stretching their inventiveness and creativity and together creating something entirely new.

‘Paint by numbers’ in the AI age

Artificial intelligence — the tech that gives computers the ability to learn — has been around since the 1950s. But over the last decade the field has enjoyed a renaissance made possible by the huge amount of data available online and the higher computational power of chips. Advances in the field over the last 10 years have enabled computers to analyze datasets and find useful patterns to solve problems, with the machine often outwitting the human brain. Creativity, however, has remained mostly in the domain of artists.

Now, software programs such as ChatGPT, DALL-E, Midjourney and Stable Diffusion are making huge strides forward in creativity. And while these tools used to be available only to researchers and a small group of invitation-only testers, an earthquake is underway: They are being publicly released and can be easily used by all, in what some say is becoming a “democratization” of creativity.

An image generated by DALL-E, a deep learning model developed by OpenAI, to the prompt: ‘Dutch football team rejoicing in World Cup win in the style of Van Gogh.’ (Image generated with the assistance of DALL-E)

On November 30, OpenAI, an AI research company co-founded by Elon Musk in late 2015 and backed by Microsoft, widely released its ChatGPT bot prototype, capable of generating sophisticated text in response to prompts and questions. The bot attracted more than 1 million users in five days, according to OpenAI’s president and co-founder Greg Brockman, wowing users with its potential to disrupt homework, journalism, code writing, academic papers, literature and more.

Other software is creating ripples in the art and design world: DALL-E, developed by OpenAI and released to the public in a beta trial in November; Midjourney, which released an open beta in July; and Stable Diffusion, which has developed a deep learning text-to-image model, also released this year.

These AI-based image-generation technologies can create people, objects and locations and mimic whole visual styles based on user requests, or prompts. Obeying simple text instructions, the latest version of DALL-E, for example, can create realistic images of a polar bear playing bass or a robot painted like a Picasso work.

The fact that these services, which once served a closed audience of researchers, are now being released to the wider public is already having “a significant impact on creativity and on culture,” said Asaf Hanuka, head of the visual communications department at Shenkar College of Engineering, Design and Art in Ramat Gan. “All of this is happening right now,” he said in a phone interview.

Shenkar, which held a conference earlier this month in Tel Aviv about creativity in the era of AI, highlighting the pros and cons of the technology, is already seeing students using these tools in their final projects, Hanuka said.

Asaf Hanuka, head of the visual communications department at the Shenkar College of Engineering, Design and Art. (Tal Shachar)

Art or artifice?

Artists have experimented with AI for their work almost from the advent of the technology.

Artist Harold Cohen was a pioneer of computer art who developed, starting in the 1970s, an art-creating program called AARON — “the world’s first pure AI artist,” according to a description of one his works at the Victoria & Albert Museum. Cohen’s software, however, was not open source and thus not publicly accessible, and AARON’s development ended with Cohen’s death in 2016.

In recent years, however, AI in the arts “has exploded,” said Drew Hemment, a professor of Data Arts & Society at the University of Edinburgh and a fellow of the Alan Turing Institute. This is due both to “advances in the technology, and the release of powerful new tools.”

“AI gives artists superpowers,” Hemment said in an email interview. “Today artists can craft images, or sound, or anything they can imagine, that harness the creative powers of artists that have gone before, and fuse human intuition with advanced computational technology.”

A screenshot from a video of art made by AARON, a robot developed by artist Harold Cohen (YouTube, used in accordance with Clause 27a of the Copyright Law)

“Today, we see artists working with AI in highly imaginative ways, as both tool and topic,” he added. “Artists are at the forefront in pushing the boundaries of the technology.”

In June this year, Cosmopolitan used DALL-E 2 to generate the world’s first “artificially intelligent magazine cover,” with a prompt that asked the software to create a “wide-angle shot from below of a female astronaut with an athletic feminine body walking with swagger toward camera on Mars in an infinite universe, synthwave digital art.”

The results were used on the cover of the magazine’s AI Issue.

In the US, Jason Allen won first prize, beating 20 other artists, for his “Théâtre D’opéra Spatial” submission at the Colorado State Fair’s fine arts competition at the end of August this year. The artwork had been created in large part using the AI tool Midjourney, but the judges couldn’t tell. This created a huge debate on the meaning of art, and Allen faced accusations of deceptiveness.

Jason Allen’s work, ‘Théâtre D’opéra Spatial’ won first prize in the Colorado State Fair’s fine arts competition in August 2022. The artwork was created in large part using the AI tool Midjourney. (Screen grab, used in accordance with Clause 27a of the Copyright Law)

“We are in the midst of a very, very rapid evolution, or maybe even a revolution, of the machines seeping into these creative art domains,” said Yoed Kenett, an assistant professor at the Faculty of Industrial Engineering & Management at the Technion- Israel Institute of Technology. “Does this mean that it’s the end of the art profession? I don’t think so. It just changes what it means to be an artist. And I think that is great.”

“Creativity is about the ability to connect, create new combinations, generate new ideas, based on old ideas, make analogies, make metaphors,” said Kenett. “That together allows us to create this magic.”

Can computers take out an artistic license?

According to Wikipedia, “art is a diverse range of human activity, and resulting product, which involves creative or imaginative talent expressive of technical proficiency, beauty, emotional power or conceptual ideas.”

So, can art made by a computer be considered art? Can a computer become the artist?

“To me, it makes no difference which tools an artist uses, I am only concerned with the quality of the art. We can make art with paintbrush and easel, or with data and algorithms. It is art if it moves us, if it is aesthetically interesting,” said the University of Edinburgh’s Hemment.

Drew Hemment, a professor of Data Arts & Society at the University of Edinburgh and a fellow of the Alan Turing Institute. (Andrew Perry)

Rather than removing the human from the equation, AI will be a new arrow in an artist’s quiver, explained Shenkar’s Hanuka.

“In creativity there are two main questions: why we do something and how we do it,” he said. “The computer does not create art. It cannot choose.” The “why” is still the sole domain of the human artist, with the machine serving as a “very eager helper and assistant.”

Art is always “a dialogue between the creator and their work tool,” he added.

Israeli artist Oren Eliav, who lives and works in Tel Aviv, has had exhibitions of his work in galleries in Israel, France and Italy, the US, and the UK among others. His first solo exhibition in the Israel Museum was inspired by 15th-century Italian painter Giovanni di Paolo’s “The Death of Lucretia.” In his work, Eliav deconstructed the original painting and created 20 large images focusing on different sections of Di Paolo’s composition.

“As an artist I always work with a DNA that is not my DNA,” he said in a phone interview. “I always ask a question in front of an image I didn’t create. My artistic work is to ask, it is a dialogue” with the work of the other artist.

During the coronavirus lockdowns in 2020, Eliav started tinkering with the image-creating AI tools available at that time. Using generative adversarial network technology, GAN — one of the first attempts to generate images, which began in the mid-2010s — he created a sequence of evolving forms, which were used as reference for his hand-painted oil-on-canvas works. The paintings, along with an original poem, became “The Moon is a Mirror” exhibition at the Braverman Gallery in Tel Aviv, which ran from January to March 2022. The project continues as a long-form video work, to be released next year.

An oil on canvas painting made by Oren Eliav, artist and lecturer at Shenkar, from the exhibition ‘The Moon is a Mirror,’ Jan-March 2022 at Braverman Gallery. (Elad Sarig)

“In my work, what I ask the machine is to start imagining. Then I match the imagination of the machine with my imagination, and something that I cannot anticipate ahead of time is created, which puts me at a very interesting point as a creator,” he said.

The AI technologies available today, he said, are an added tool in his creative kit.

“When a person is creative, and creates, they will use all of the tools that are available,” he said. “If a person is not creative and not original, even if they have all the coolest tools, they won’t be able to get an original output. These technologies are like a musical instrument: everything depends on who is using them.”

Dangers lurk on the margins

Despite the advances, these new AI technologies come with pitfalls. Bias is one of them. All AI systems depend on what information they are fed – and thus they could perpetuate any existing prejudice and biases they are trained with. Facial recognition software, for example, has come under fire by civil liberties activists for being biased against people of color because of skin-type and gender bias intrinsic to the systems.

To prevent misuse or the perpetuation of violence, curbs have been inbuilt into generative AI software. For example, OpenAI’s ChatGPT uses a Moderation API to warn against or block certain types of unsafe content. The firm has also limited DALL-E 2’s ability to generate violent, hateful or adult images, and political content, according to its website.

But this limits the software’s creativity, said Hanuka, since art does depict violence and death and is often a political statement. “’Guernica’ of Picasso is an image of war,” he noted.

In addition, Hanuka said, because these generative AI tools collect and synthesize information from existing content, the natural inclination for them will be to work with images and items that are familiar and mainstream — which may lead to predictable, boring results.

“Algorithms prefer what is familiar, and the margins tend to disappear,” Hanuka said. “But actually, many times, innovation comes from the margins.”

There is also the issue of intellectual property. AI art creates images using models that have been trained on the original work of other people and artists. This raises ethical and practical questions.

“One major concern is that current industry standard AI models have been trained on massive datasets scraped from the internet such as images, or news. That happens without permission, or awareness, or fair pay for the original creators. This still needs to be tested in the courts, but it is obviously plain wrong,” Hemment said.

“I expect to see a new generation of models trained on licensed images, but that brings a new problem, as only a small number of big players have the scale to do that. That leads to further centralization, with an ever-decreasing group of companies calling the shots.”

The internet and social media were hailed early on for their ability to democratize information and make the world more connected. Yet both have also enabled privacy infringement and political manipulation.

The lessons learned in the past 20 years must now be used to make sure legal frameworks are in place to tackle generative AI, said Tehilla Shwartz Altshuler, a senior fellow at the Israel Democracy Institute and an expert in law and technology.

“We know today that those different platforms that allow us to create our own content must have some kind of responsibility for the outcome created on top of them,” she said. “This is what is happening today with social media platforms, but the next frontier is going to be the generative AI platforms. These platforms are not going to grow up in a vacuum, because this is 2023, and not 2000. There is a lesson we have learned in the past two decades.”

Curbs on pornography, terror and violence should be enforced on these platforms, she urged, and detection systems should be created to help us sort truth from lies and to show us where, when and from what source a certain piece of content was created.

The wow factor

For the sake of testing out the technology, this reporter used Dall-E to create two images. After signing with a username and password, I put in the following prompts: Woman journalist at desk working at laptop with a cup of coffee. In just a few seconds, four images were created, one of which is shown here.

An image created by DALL-E, a deep learning model developed by OpenAI, to the prompt: ‘Woman journalist sitting at her desk with a laptop and a hot mug of coffee in cubism style,’ December 2022. (Image generated using DALL-E with a prompt by by Shoshanna Solomon)

The second prompt, to make my Dutch husband happy, was the following: Dutch soccer team rejoicing in the win of the World Cup in the style of Van Gogh. That image, which I plan to give him as a consolation prize for his team’s loss, is also reproduced here and was also created in seconds.

So, whatever hidden dangers these new technologies may hold, it is clear that AI is here to stay in the creative sphere. Just as we have largely ceded our navigation abilities to Waze and our information gathering to Google, so will we hand over greater chunks of the creative process to the machine.

The technology will allow a greater number of people to be creative, said the Technion’s Kenett, giving them tools to express themselves even if they don’t know how to write well or paint well, allowing for a democratization of creativity.

And the issue of whether or not AI was used in a given work of art will quickly become irrelevant, said Shenkar’s Hanuka.

“Did you use a machine, yes or no, is something that is not relevant and will disappear with time,” said Hanuka. “It doesn’t really matter. What matters is the results you get. A creation that will have success, which creates impact, is what we will remember.”