APPARATUS, SYSTEMS AND METHODS FOR AN EFFECTS RENDERING PIPELINE, INCLUDING MAKEUP EFFECTS

Smart overview of the Invention

The patent application focuses on enhancing the process of applying effects, such as makeup, to objects in video frames. This involves performing effect rendering and object landmark determination simultaneously. The technology finds applications in virtual try-on (VTO) for products like makeup, video conferencing, and teleconsultation. The system aims to improve user experiences by efficiently applying effects in real-time video scenarios.

Field of Invention

The invention pertains to image processing technologies, particularly those used in virtual try-on applications, augmented reality (AR), and virtual reality (VR). It is designed for applications like video chat, teleconsultation, and video conferencing, where visual effects are applied to enhance user interaction. The focus is on creating an efficient pipeline for rendering effects such as makeup.

Background

Current VTO technologies allow users to apply virtual makeup onto photos or live videos. This is useful in e-commerce for trying products before purchase and in video conferencing to maintain a polished appearance. Challenges arise due to facial movements, occlusions, and the need for high frame rates with low latency. These factors can disrupt the realism of the effects, requiring robust solutions for seamless application.

Summary

The invention introduces an improved augmented reality makeup pipeline that addresses these challenges by optimizing the effects rendering process. It performs effect rendering in parallel with determining object landmarks, using advanced techniques like deep neural networks for precise tracking. This approach ensures smooth application of effects across video frames even during movement or occlusion.

Detailed Description

The system utilizes a double-threaded pipeline for efficient processing, as opposed to traditional single-threaded methods. This involves using deep neural networks to track facial features and apply effects accurately. The process includes detecting occlusions and adjusting effect rendering accordingly to maintain realism. The technology supports multiple objects within a frame, allowing diverse applications such as virtual try-ons for different facial regions.