Have you ever wondered how you would look like if your portrait is painted by the greatest of the great artists in history like da Vinci, Vermeer, and Picasso? Now, you don’t need to time travel and become a part of the royal court for your photos to be rendered in the style of the greatest maestros of all time.
Researchers from MIT’s IBM Watson AI Lab have developed a new photo manipulation project called AI Portraits that can transform your photos into Rennaissance art with just a click of a button. Using artificial intelligence (AI) based on generative adversarial network (GAN), AI Portraits is virtually reconstructing a photo uploaded in its systems and rebuilding them pixel per pixel to render the submitted photo in the style of an auto-generated art master in the past.
It’s different from other photo-editing tools
But this is not the same as other photo-editing technology. Unlike Neural Style Transfer technology used by common photo manipulation app available in the market right now, with “strong alteration of colors, but the features of the photo remain unchanged,” AI Portraits “creates new forms, beyond altering the style of an existing photo.”
“AI Portraits Ars can paint portraits in real-time at 4k resolution. You will find yourself in front of a mirror and feel thousands Rembrandt, Caravaggio, Titian portraying you moment after moment,” reads AI Portraits website.
How does it work?
AI Portraits Ars uses Artificial Intelligence to reproduce artistic human portraits, with different styles and levels of abstraction. The platform uses the generative adversarial network (GAN) that are “trained to learn a mapping of a latent vector z ∈ Z to a generated image y = G(z) with G being the generator.” The latent space Z describes all possible portraits. “AI Portraits Ars pushes us towards an intuitive and playful way of interacting with state-of-the-art GAN models.”
“By showing our face to the neural network, we walk through the Z space and identify the vector that best describes our face in the multidimensional space of the GAN. We trained AI Portraits Ars using our GAN on 45,000 portrait images. To allow insertion of own images into the latent space of a model, we trained an inverter that can approximate the latent vector z = I(x) from an image x,” explained the developers of the platform.
Focus on European art
In training the network, the developers have used thousands of images across different art movements from the Early Renaissance to Contemporary Art. The developers said that they have a massive dataset of artworks that crosses cultural boundaries and epochs, but they focused on 15th century Europe, which is considered by art historians as “a stylistic inflection point in the history of portraiture marked by the emergence of realistic depictions of individuals.”
Interestingly, the developers have said that they are still testing certain biases in the AI. For one, they have trained their machine to recognize certain preferences from a work of art and render them to the images they produce. One, in particular, is smiles. Many of the masters in classical art do not paint smiling portraits as smiling is viewed as informal during their times. As a consequence, they have trained their AI to decide whether to render a smiling photo into an image with a smile.
“Training our models on a data set with such strong bias leads us to reflect on the importance of AI fairness […] AI Portraits Ars introduces a very different type of bias with unique themes to explore,” the developers explained.
“We encourage you to experiment with the tool as a way of exploring the bias of the model. For example, try smiling or laughing in your input image. What do you see? Does the model produce an image without a smile or laugh? Portrait masters rarely paint smiling people because smiles and laughter were commonly associated with a more comic aspect of genre painting and because the display of such an overt expression as smiling can seem to distort the face of the sitter. This inability of artificial intelligence to reproduce our smiles is teaching us something about the history of art,” they wrote.
If in case you are worried about your photos being uploaded on the internet after the spectacle made by FaceApp in the past few days, AI Portraits said that “your photos are sent to our servers to generate portraits. We won’t use data from your photos for any other purpose, and we’ll immediately delete them.”