EXTRACTION OF HUMAN POSES FROM VIDEO DATA FOR ANIMATION OF COMPUTER MODELS

Smart overview of the Invention

The application introduces a method for extracting human poses from video data to animate computer models. This involves determining a sequence of human body poses from video frames, capturing movements of a person. The initial pose sequence undergoes updates through multiple iterations to refine the animation quality. The method focuses on enhancing animation accuracy while reducing computational demands, making it suitable for various applications, including games and movies.

Technical Process

The process begins with determining a first pose sequence from video frames using a machine learning model. This sequence is refined into a second pose sequence through iterative adjustments. Joint velocities are calculated, and predicted poses are generated in parallel for each pose in the sequence. By comparing these predicted poses, joint angles are adjusted to improve the animation's fidelity to the original movements depicted in the video.

Implementation Details

Various implementations are described, such as determining predicted poses in chains and adjusting based on consistency errors. The process can include encoding joint velocities into a human motion prior and decoding them to predict poses. Adjustments may involve adding delta velocity parameters and refining them through comparisons. The update process can stop when deviations meet predefined thresholds, ensuring efficiency.

Model Training and Error Correction

The initial pose sequence is derived using a machine learning model trained on numerous videos. This sequence is refined by projecting poses from 3D to 2D, calculating reprojection errors, and iteratively adjusting parameters. Additional errors like off-balance orientations or excessive joint velocities are also considered in adjustments. Cost functions are applied to maintain realistic human body shapes and movements.

System and Application

A system with a processor and memory executes the method, performing operations to determine and update pose sequences. Constraints, such as camera movement limitations, are applied during adjustments. The resulting pose sequence animates a computer model, closely matching the person's movement in the input video, offering a practical solution for creating high-quality animations efficiently.