Is the era of in-store shopping over? Technology is taking the beauty industry to a whole new level.
Throughout the years, the makeup industry has transformed and grown exponentially. Brands provide a wide range of products appealing to different skin types and individual preferences. However, as the number of products has increased, it has become challenging to determine which product addresses one’s unique needs.
The product descriptions offered by traditional e-commerce sites are quite inadequate. This situation stems from the lack of information about the material properties, such as the exact color, level of glitter, and finish type, which are vital when it comes to purchasing a makeup product.
To solve this problem, Dezaki et al. (2023) developed a pipeline, identifying the material properties of the makeup product from images. This tool is beneficial for both the discovery of products and the experience of virtual try-on systems, which optimize the buying process by making it more engaging and precise.
The automated pipeline introduced by researchers consists of three elemental components:
- Makeup Material Property Extraction: The function of this component is to identify the main features of the product, including base and reflective color, format, and finish type from makeup images. Format refers to the texture of the product—powder, cream, stick, or liquid—whereas finish type is the effect that makeup products leave on the surface, such as matte, shimmer, metallic, and glitter. Multiple machine learning methods were leveraged, involving the selection of images, detection of shade regions, color estimation, and format classification.
- Clothes Material Property Extraction: Properties like color and texture are extracted from the apparel photos using this component. These characteristics are used to suggest certain makeup items that go well with the apparel.
- Makeup Match Maker: This component is useful for recommending particular makeup products based on individuals’ preferences or apparel. The pipeline is capable of generating tailored recommendations like complementary eyeshadow or lipstick with specific clothing.
Researchers have done extensive experiments, encompassing a data set of 3,700 eyeshadow products. Per product, there are approximately 5 images. The tool performs effectively in identifying material traits, according to the results. The base color identification was accurate in a way that, for humans, it was difficult to make a differentiation between predicted and actual colors.
Additionally, the pipeline has demonstrated a robust effectiveness and adaptability in other categories of makeup products like lipstick and foundation, proving its ability to function in a variety of makeup products.
Furthermore, to assess the performance of their machine learning methods compared to human annotators, the scientists carried out other tests, resulting in consistent and more objective results than human judgment. These outcomes show that the pipeline is pretty accurate and reliable.
The Realm of Product Recommendations
In the scope of product recommendations, the authors’ approach is proven to be effective. The system makes recommendations that are more pertinent and tailored by integrating material properties of the product. For instance, the recommendation system is able to suggest a eyeshadow or a lipstick, complementing the color and finish type of a particular apparel. A user study suggested that 78% of the participants have chosen the recommendations provided by the system, demonstrating the significance of material features in makeup recommendations.
Takeaways
In this research, an extensive pipeline of machine learning for identifying the material attributes of makeup products is presented. This tool is effective not only for product discovery but also for virtual try-on experiences. It is quite accurate in a wide array of makeup categories, including eyeshadow, lipstick, and foundation. Moreover, relevant suggestions of clothing are enabled by this tool, highlighting the importance of material attributes for personalized recommendations. Although virtual try-on systems are widely used among many brands in the makeup industry, this tool has the potential to transform the online makeup shopping experience by providing more accurate and personalized product suggestions.
Reference
Dezaki, F. T., Arora, H., Suresh, R., Banitalebi-Dehkordi, A., & Celikik, M. (2023, September). Automated material properties extraction for enhanced beauty product discovery and makeup virtual try-on. In Workshop on Recommender Systems in Fashion and Retail (pp. 33-52). Cham: Springer Nature Switzerland.
