US20260045020
2026-02-12
Physics
G06T13/40
The patent describes a method for creating a realistic and animated facial avatar using a Head Mounted Display (HMD) device. It involves capturing multiple facial images of a user from predefined perspectives. These images are then processed to generate perspective embedding vectors that represent the user's facial expressions. A pre-fed neutral facial image is also used to generate neutral embedding feature vectors, which help capture the user's identity without any expression.
The HMD device utilizes an AI/ML-based expression transfer model to correlate perspective embedding vectors with neutral embedding vectors. This correlation results in a frontal facial image that accurately reflects both the identity and expressions of the user. Further, the HMD performs 3D morphing on this frontal image to create a detailed and realistic facial avatar.
For generating animated avatars, the HMD captures facial images from multiple perspectives and encodes them into perspective embedding vectors. These vectors help generate Action Unit (AU) values and associated uncertainty values that predict expression coefficients. These coefficients are applied to an animated avatar, allowing it to mimic the user's expressions accurately.
The method addresses limitations of conventional techniques by allowing for higher resolution and more accurate avatar representations. It uses a deep neural network model to create expression clusters, improving the similarity score between different vectors. The process iteratively refines the avatar's resolution until a predefined threshold loss is achieved, ensuring a high-quality final product.
The HMD device includes a processor and memory that store instructions for executing the avatar generation process. It supports switching between non-animated and animated avatar modes based on user input, providing flexibility. The device's image capturing components are synchronized to ensure consistent and accurate data collection from various perspectives.