METHOD AND SYSTEM FOR 3D HAIR VIRTUAL TRY ON USING PHYSICS SIMULATION

Smart overview of the Invention

A novel system for 3D hair virtual try-on (VTO) leverages physics simulation to replace a user's natural hair with a realistic 3D representation. By simulating physical forces, the system deforms the 3D hairstyle to mimic real hair behavior. This approach captures detailed textures and applies color independently of texture, allowing for accurate color representation. The system also adapts hair color based on ambient light conditions and supports advanced coloring techniques like highlights, contouring, and balayage by segmenting hair strands into groups.

Background

Realistic hair recoloring during real-time VTO is complex due to the intricate nature of hair and the detail required for realism. Traditional methods rely on pixel recoloring, which often fails when input images lack detail, especially in darker shades. These techniques struggle with advanced coloring styles that require different treatments for various hair portions. Existing technologies cannot reliably track individual hair strands across video frames, limiting their effectiveness in dynamic VTO applications.

Technical Approach

The system employs a machine learning model to maintain the user's hairstyle while altering its color to match specific products. It uses intuitive parameters for controlling color placement, shine, and intensity, allowing multiple colors simultaneously. A physics simulation ensures hair strands move naturally in response to head movements and environmental interactions. Additionally, an environment map estimation enhances realism by lighting the 3D hair mesh according to surrounding conditions.

System Components

Key components include a 3D face detector, hair segmentation module, environment map estimator, and hairstyle detector. These elements work together to process input images or video frames, replacing detected hair with a 3D model while preserving the user's original hairstyle. A renderer processes outputs from these components to generate realistic VTO effects by removing detected hair and applying a bald filter or suggesting tight hairstyles to enhance rendering quality.

User Interaction and Implementation

Users can interact with the system through interfaces that allow selection of hairstyles or colors and recommend VTO options. The system can also facilitate transactions for purchasing associated products. Implemented on devices like smartphones or tablets, it processes input images using stored components to deliver a seamless VTO experience. The technology adapts to user preferences while ensuring high-quality rendering through advanced simulations and environment-aware lighting.