SYSTEMS AND METHODS FOR EFFICIENT RENDERING PIPELINE FOR MAKEUP INCLUDING VIDEO CONFERENCE STREAMLINING

Smart overview of the Invention

The patent application introduces a system and method to efficiently apply visual effects, such as virtual makeup, to objects in video frames. This technology is particularly relevant for video chatting, conferencing, and teleconsultation. The process involves parallel operations for rendering effects and determining object landmarks, enhancing the performance and realism of virtual try-on (VTO) applications. The system is designed to address challenges like face movement, occlusion, and maintaining high frame rates with low latency.

Field of Invention

The invention pertains to image processing and its applications, including virtual try-on (VTO), augmented reality (AR), and virtual reality (VR) applications. It focuses on improving the efficiency of effects rendering pipelines used in video communication platforms. The technology can be integrated into various applications such as e-commerce for virtual makeup trials, video conferencing with virtual effects, and teleconsultation.

Background

Virtual makeup technology allows users to apply makeup digitally during video calls or conferencing. However, challenges arise from face movements, occlusions, and ensuring a seamless experience with high frame rates. These issues necessitate robust face tracking and effect rendering technologies that can adapt to various conditions like lip movements and partial occlusions.

Summary of the Invention

The proposed system enhances the efficiency of augmented reality makeup pipelines by performing effect rendering and landmark detection in parallel. This approach improves the application of effects in real-time video frames, making it suitable for VTO in video chats or teleconsultations. The system includes a user interface for selecting various effects and an e-commerce interface for purchasing products.

Technical Implementation

The invention uses a parallel processing method to apply effects on video frames efficiently. Techniques like optical flow functions are employed to predict landmark locations, enhancing stability and reducing processing time. The system can detect occlusions and adjust effects accordingly, ensuring realistic application. Additionally, it integrates with video chat applications to offer product recommendations and facilitate purchases through an e-commerce platform.