In 2018, a purchase that now seems unthinkable was made. Someone paid over $400,000 for an artificial intelligence (AI) piece of art. The painting, called “Edmond de Belamy,” was a portrait of a gentleman dressed in black. It looked a bit like a painting you would see in a museum, except it looked a bit unfinished and had a mathematical equation instead of a signature.

The reason why this seems unthinkable now is because we can create strikingly similar images (probably even better) within a minute.

Several solutions (including some free ones) can create realistic images; in fact, some are so good that in 2022, one such AI famously won an art competition. Since then, AI has progressed even more, up to the point where AI art has become a comprehensive field.

While some artists have embraced this as a complementary type of fine art, others are understandably worried. Suddenly, what once required years of training and honing one’s craft can now be achieved with a few clicks. This democratization of art creation has opened up new possibilities for creativity and expression, but it also raises questions about the value of human creativity and the future role of artists in society.

This is not just about creating art, either. AI has dipped its proverbial toes into the art world in several ways. Here are some examples.

Data Enters the Art World

Art forgeries are strikingly common. According to some experts, around half of the art market consists of forgeries, but this does not stop massive sums being paid for artwork. Even leading experts get fooled sometimes. To combat this phenomenon, AI is now being utilized to detect forgeries and determine the artist behind undocumented paintings.

Algorithms can scrutinize and analyze artworks with techniques to identify telltale signs that are imperceptible to the human eye. They can look at brushstroke patterns, color distribution, even the specific composition of materials used. This data-driven analysis allows AI to compare a suspect painting against a vast database of authenticated works, detecting inconsistencies or anomalies that may indicate a forgery.

This is how, for instance, a painting called “de Brécy Tondo” was revealed to be a work by Raphael, the famous 16th-century High Renaissance artist. For over a century, “de Brécy Tondo”was considered to be a copy of a work by Raphael and not an original one. But Hassan Ugail, a professor at the University of Bradford in the UK, developed an artificial intelligence model that can identify paintings created by prolific painters like Raphael.

“My AI models look far deeper into a picture than the human eye, comparing details such as the brush strokes and pigments. Testing the Tondo using this new AI model has shown startling results, confirming it is most likely [97%] by Raphael,” Ugail told the Guardian.

Some art experts agreed with the AI results. Others did not. Things became further complicated by the results of another AI trained for this purpose, which concluded that the painting is most likely not from by Raphael.

So the experts do not agree, the AI do not agree, and we run into an already classic AI problem: It is not clear why exactly the AI decides what it decides.

Training Algorithms

Machine learning algorithms involve millions or billions of different parameters that interact, often in an intricate, non-linear, and non-intuitive fashion. These systems learn from vast amounts of data and make inferences that are not explicitly programmed by humans, which often results in “black box” models where the output is clear, but the internal working that led to the output is not.

This lack of transparency is not limited to the realm of AI art – numerous AI applications, including healthcare and criminal justice, where understanding the basis of AI decisions is crucial for trust, ethics, and accountability are affected as well. The case of AI in art, however, is particularly challenging due to the lack of training data.

Typically, many robust models use thousands of images for training. While he may be considered one of the most prolific artists in history, Raphael painted ‘only’ around 500 works of art. Granted, models can be trained with smaller datasets, but this can be a significant limitation in how accurately AI can detect forgeries and the artists behind paintings.

Selecting the dataset also becomes much more challenging in this scenario. If we were to choose a dataset that included, say, paintings by Raphael and paintings by Picasso – which are vastly different in style – the resulting model would likely have no problems differentiating between the two. However, if the algorithm were trained only on works by Raphael and his disciples, the algorithm would have problems generalizing and applying the same analysis to real-world data.

So the problem of AI accuracy in identifying paintings is still an open one. In the meantime, however, AI is already being used in the real world. If that sounds familiar, well, it is not too dissimilar from other areas in which AI is applied.

Then, there is perhaps the most spectacular application of all: completing paintings.

AI, Please Complete This

As AI algorithms can learn from existing data and extrapolate patterns, styles, and techniques, they can also generate pieces that seamlessly blend with an artist’s style.

In other words, this means AI can complete and expand existing artwork by keeping the work’s original aesthetic, making informed guesses about the missing parts and ostensibly aligning it with the original artist’s vision. At least that is the theory.

This was famously demonstrated in 2019, at the Rijksmuseum in Amsterdam. The museum curators embarked on a restoration of Rembrandt van Rijn’s 1642 masterwork, “The Night Watch.”

The painting originally measured 363 cm by 437 cm. However, in 1715, to accommodate its placement between two doors at Amsterdam City Hall, the painting was significantly trimmed, losing parts of its original dimensions and, consequently, elements of its intended composition and dynamism.

Leveraging machine learning, the museum used a high-resolution scan of the existing painting along with a pre-trim copy of the painting. The copy had a slightly different perspective than the original and used somewhat different brush strokes, whereas the AI restoration returned Rembrandt’s original dynamic perspective and style. The AI-generated additions were printed and displayed alongside the original.

This was a special case, in which a copy existed that aided the AI reconstruction. Another example of reconstruction is mending the passage of time.

Researchers at TU Delft are pioneering the use of deep learning techniques to digitally reconstruct Vincent van Gogh’s artworks to understand their original appearance before time-induced fading. This endeavor, highlighted by the example of “Snow-Covered Field with a Harrow (after Millet)” in the Van Gogh Museum, aims to recover the original hues and details obscured over the years.

There is even yet another way in which AI is furthering painting reconstruction.

Utilizing neural network technology, researchers have digitally uncovered an image hidden beneath Picasso’s “The Old Guitarist,” a piece from his Blue Period. The artist, like many other painters, had a habit of painting one image over another; now, the painted-over image has been reconstructed. The reconstruction was based on an X-ray, but the X-ray revealed only the general geometry of the lost painting, it was the AI that reconstructed it.

But can we truly recover lost paintings using AI?

The answer depends on what you mean by “recover”. It is subjective, we have only a few sample points, we are unsure what the original artists actually envisioned – there are issues up for debate.

Where Does All This Lead Us?

The rapid advancement of AI presents a double-edged sword for the art world. On one hand, AI has democratized the creation of art, enabling the production of artwork that arguably rivals the quality of pieces which traditionally required years of human skill and creativity. This technological leap has opened new avenues for creativity and expression, making art creation more accessible to a broader audience.

However, this shift also prompts a reevaluation of the value of human creativity and the future role of artists, challenging traditional notions of artistry and the unique value of human-made art. As AI art becomes increasingly prevalent, the art community is divided, with some seeing it as a complementary evolution of the artistic process and others viewing it with skepticism and concern for the implications on human artists.

AI has shown undeniable promise in detecting art forgeries and uncovering the origins of undocumented artworks, tasks that require analysis beyond human perceptual limits. Yet, this application is fraught with uncertainties due to the opaque nature of AI decision-making processes and the limitations of training data.

At first glance, you would probably expect art and AI to have few things in common; and yet, the implications we are already seeing show that AI in art is essentially a microcosm of AI in society. It reflects both its vast potential and the complexities it introduces, prompting complex discussions that came out seemingly out of nowhere.

Finally, these discussions are not only limited to art. We are all in the midst of a collective conversation on how we integrate AI into society, ensuring it enhances rather than diminishes our human experience. Hopefully, we can find the right answers.