METHOD AND SYSTEM FOR 3D HAIR VIRTUAL TRY ON

Smart overview of the Invention

The patent application presents a system for 3D hair virtual try-on (VTO) that leverages advanced image processing techniques. This system replaces a user's natural hair with a 3D representation that closely mimics real hair behavior. The 3D model incorporates physics simulations to account for forces such as gravity and head movement, enhancing realism. The technology also adapts to ambient light conditions, ensuring the rendered hair color is accurate and dynamic.

Challenges and Innovations

Traditional VTO systems face challenges in achieving realistic hair recoloring due to hair's organic nature and the need for detailed texture. Existing methods often fail with darker hair or complex styles like highlights and balayage. This invention overcomes these limitations by segmenting hair into groups of strands, allowing for precise recoloring effects. It employs machine learning to maintain hairstyle consistency across video frames, a significant improvement over previous technologies.

Technical Implementation

The system uses a combination of components for processing input images, including a 3D face detector, hair segmentation tools, and environment map estimation. These components work together to create a 3D hairstyle model that matches the user's existing style while allowing color changes. A machine learning model estimates environmental lighting to ensure the 3D hair mesh is lit accurately, providing a natural appearance.

User Interaction and Customization

Users can interact with the system through interfaces that allow selection of hairstyles and colors, offering personalized VTO experiences. The system can recommend options based on user preferences and even facilitate transactions for purchasing related products. It supports multiple colors simultaneously and provides intuitive controls over color placement, shine, and intensity.

System Architecture

The application runs on consumer devices like smartphones and tablets, utilizing their cameras for real-time video processing. Key components include a renderer that integrates outputs from various detection modules, enhancing realism by removing detected real hair and applying a bald filter or alternative styling instructions to optimize the 3D hairstyle rendering. The system's flexibility allows users to either follow suggested styling adjustments or proceed with their current look.