Invention Title:

MULTIMODAL CONDITIONING FOR IDENTITY-PRESERVING MOTION SYNTHESIS

Publication number:

US20260340922

Publication date:
Section:

Physics

Class:

G06T13/40

Inventors:

Assignee:

Applicant:

Smart overview of the Invention

The patent application describes a method for generating identity-preserving motion sequences for avatars in virtual environments. This is achieved by performing a denoising diffusion process, which is conditioned on a motion content input and an identity profile. The identity profile includes attributes such as body shape parameters, demographic traits, and occupational characteristics. The process begins with a noise vector and iteratively generates denoised video frames, each containing a pose, until a termination criterion is met. The resulting motion sequence maintains the character's identity while reflecting the desired motion.

Technical Field

The invention pertains to the field of motion generation for virtual characters, specifically avatars, using multimodal conditioning. It involves methods, systems, and computer-readable media that utilize diffusion-based generative AI models to create motion sequences that preserve the character's identity. The approach addresses the challenge of creating realistic character movements in virtual environments, which is particularly difficult for non-human characters due to a lack of training data.

Background

Character movement in virtual environments often appears unrealistic, with jerky or unnatural motions. This is especially true for non-human characters or those with unique attributes, as machine learning models struggle to predict plausible movements due to insufficient training examples. The application aims to overcome these limitations by employing a diffusion model that generates smooth and identity-consistent motion sequences.

Methodology

The method involves obtaining a motion content input, which specifies a target motion sequence, and an identity profile for the character. The denoising diffusion process is carried out using a motion diffusion model over multiple timesteps, starting with a noise vector. Each video frame generated during the process includes a character pose. Once the process meets a termination criterion, the final set of video frames is output as an identity-preserving motion sequence. This sequence accurately reflects the character's identity and the target motion.

Implementation

The system comprises processors and memory that execute instructions to perform the described operations. The motion content input can be a natural language text prompt or a sequence of keyframe poses. The identity profile can include various attributes, such as demographic or occupational traits, and is encoded separately from the motion content. The diffusion model may utilize transformer encoders and multilayer perceptrons to process these inputs. The approach enables the generation of realistic and identity-consistent motion sequences, even for characters with unique or non-standard attributes.