ANIMATING VIRTUAL AVATAR FACIAL MOVEMENTS

Smart overview of the Invention

The patent application describes systems and methods for animating virtual avatar facial movements by generating animations that correspond to a subject's pose. This involves accessing image data of the subject's face and processing it by analyzing subregions in a biologically ordered sequence. The process starts with larger scale movements and progresses to smaller ones, using optimization techniques to fit animation rig parameters to the image data, resulting in an avatar that mimics the subject's pose.

Technical Field

The invention is situated within the realm of virtual reality (VR), augmented reality (AR), and mixed reality (MR) technologies. It focuses on improving the animation of virtual characters or avatars by utilizing advanced rigging systems. The goal is to enhance the interactive experience in VR, AR, and MR environments by providing high fidelity digital avatars that closely replicate real-world facial expressions.

Background

Modern computing advancements have enabled immersive VR, AR, and MR experiences where digital images appear realistic to users. This technology blends virtual images with real-world inputs in varying degrees. The disclosed systems address challenges in these fields by offering solutions for animating avatars with high fidelity, ensuring they accurately represent human expressions and emotions using facial mapping techniques.

Summary of Techniques

The system employs a Facial Action Coding System (FACS) to classify facial expressions based on muscle movements, represented as Action Units (AUs). The process involves adjusting facial rig parameters for each subregion to minimize discrepancies between the subject's image and the animation rig. By iteratively solving for each subregion in a specific order, from large-scale movements like the jaw to finer details like the eyes, the system creates a realistic avatar representation.

Implementation Details

The Facial Solver is a key component that processes 3D scans of subjects captured using multiple synchronized cameras. It converts these scans into animation rig data by iteratively adjusting subregions until an accurate match is achieved. This hierarchical approach allows for efficient correlation between different data sets, ensuring that avatars can display complex expressions such as smiling or frowning with high precision.