SYSTEM AND METHOD FOR MACHINE-LEARNING BASED MODIFICATION OF USER IMAGES BASED ON REFERENCE IMAGE OR GRAPHICAL AVATAR

Smart overview of the Invention

The patent application describes a computer system designed to modify user images by leveraging machine learning to adapt features from a reference image or graphical avatar. The system extracts attributes from both a reference image and a target user image, processes these through a machine learning model, and outputs modification data to create a modified version of the user's image. This process is facilitated by a virtual try-on engine that applies the derived modifications to the target image.

Machine Learning Model

The machine learning model is trained using a supervised learning approach, utilizing a dataset of modified user images labeled with avatar attributes. The reference attributes may include color values, such as hexadecimal color codes, which are used to generate corresponding color modifications for the target image. For example, hair color values from a graphical avatar can be used to alter the hair color in the user's image.

Modification Attributes

Reference attributes can include face color values, which are used to modify the face area of the target image, potentially applying virtual cosmetics that match the reference image's face color. Texture values from the reference image can also be used to adjust the texture of the target image, such as applying a cosmetic finish. The system is capable of modifying other features, like hair and eyelashes, by using attributes such as length, density, color, and texture from the reference image.

Customization and Recommendations

The system supports user interface features that present product or style recommendations based on the extracted attributes and the modified user image. This functionality can help users find products that match their desired look, offering additional features like tutorials tailored to the user's cosmetic skill level. The system aims to reduce the time users spend searching for products to achieve specific looks by providing accurate matches based on user attributes.

Technical Implementation

An illustrative example demonstrates the system's capability to customize avatars using user images. The process involves detecting facial features, segmenting the image into regions for analysis, and utilizing machine learning to apply modifications. The system can generate a geometric face map to ensure accurate transposition of looks from a reference image to a user's unique facial geometry. Augmented reality can be employed to overlay these modifications on live images, enhancing the user's experience with real-time visualizations.