A model that uses human inspiration and designs to create realistic fashion images

Artificial Intelligence (AI) has recently begun to enter many creative industries. In these contexts, AI can automate tedious or time-consuming processes, as well as empower artists and facilitate their creative process.

Researchers from the University of Florence, the University of Modena and the University of Reggio Emilia and Pisa plan to soon explore the potential of AI models in fashion design. On a pre-printed paper arXivintroduced a new computer vision framework that helps fashion designers visualize their designs by showing how they look on the human body.

Most of the past studies exploring the use of AI in the fashion industry have focused on computational tools that can recommend clothes similar to those chosen by the user, which can show online customers how the clothes look on their body (i.e., virtual try-on systems). This group of Italian researchers has developed a framework that supports the work of designers by showing how the clothes they design look like in real life to find new inspiration, identify potential issues and change their designs if necessary.

“Different from previous works that mainly focused on the virtual experiment of clothing, we propose a multimodal conditioning fashion image editing work, following such as text, human body position, such as text, human body position, etc., and lead the generation of human-centered fashion images and clothing images” Alberto Baldratti, David Morelli and colleagues wrote in their paper.

“We solve this problem by presenting a new architecture based on latent diffusion models, an approach not previously used in the fashion domain.”

Instead of using artificial neural network architectures/generative adversarial networks (GANs), which are often used to generate new text or images, the researchers decided to create a framework based on latent distribution models, or LDMs. As they are trained on compressed and low-dimensional latent space, LDMs can generate high-quality synthetic images.

Although these promising models have been applied to many tasks requiring the generation of synthetic images or videos, they have been rarely used in the context of fashion image editing. Most of the previous works in this area have introduced GAN-based architectures, which produce lower quality images than LDMs.

Most existing datasets for training AI models on fashion design tasks only include low-quality clothing images and do not include the necessary information to create fashion images based on text queries and patterns. In order to effectively train their model, Baldratti, Morelli and their colleagues first had to update these existing datasets or create new ones.

“In the absence of existing datasets suitable for the task, we extend two existing fashion datasets, Dress Code and VITON-HD, to semi-automated multimodal annotations,” Baldratti, Morelli and colleagues explain in their paper. . “Experimental results on these new data sets demonstrate the effectiveness of our proposal in terms of realism and multimodal input.”

In initial evaluations, the model created by this group of researchers has achieved very promising results in creating realistic images of clothing on the human body and based on specific textual suggestions. The source code for their model and the multimodal annotations they added to the datasets will soon be released on GitHub.

In the future, this new model can be integrated into existing or new software tools for fashion designers. It can also inform the development of other AI architectures based on LDM for real-world innovation applications.

“This is one of the first successful attempts to imitate the work of designers in the creative process of fashion design and could be a starting point for the control of distribution models with human input in creative industries,” Baldrati, Morelli and their colleagues conclude in their article.