GENERATING THREE-DIMENSIONAL LOOPING ANIMATIONS FROM STILL IMAGES

Smart overview of the Invention

The disclosed system leverages neural networks to transform still images into looping 3D animations. By fitting a 3D model to a person's pose in a digital image, the system processes animation sequences that transition between starting and ending poses. Using an animation transition neural network, it creates two 3D animation transition sequences that bridge the initial pose with the starting and ending poses. Texture mapping is applied to these sequences to generate a seamless looping animation.

Background

Advancements in image animation have made editing software more accessible, yet transforming 2D images into realistic 3D animations remains challenging. Current methods like 2D rigging often fail to produce physically plausible results. Creating a looping 3D sequence from a specific initial pose is difficult due to limitations in flexibility and accuracy. Existing techniques struggle with rendering realistic secondary motions, such as clothing dynamics, within animations.

System Features

The system integrates an animation transition neural network with an animation rendering neural network for realistic looping animations. It employs a 3D body model to depict complex body movements and uses a neural in-betweening method for non-linear motion interpolation. This allows for accurate modeling of human motion loops. Additionally, the system synthesizes high-quality appearances with motion-dependent textures, enhancing visual appeal while maintaining efficiency.

Detailed Functionality

Focusing on human-specific animations, the system estimates a 3D model from a still image and aligns it with a target motion sequence. Through deep learning, it creates two transition sequences that connect the model's initial pose with the target sequence's start and end points. These are combined into a cohesive loop, rendered with textures specific to the individual in the image. Minimal user input is required—only an initial image and desired animation sequence—making the process automated and user-friendly.

Advantages Over Conventional Systems

Unlike traditional methods that often yield implausible animations, this system uses a 3D body model for realistic movement representation. It efficiently synthesizes secondary motions like clothing dynamics using neural rendering techniques. The system's compact motion representation conserves computational resources while maintaining high fidelity in rendering time-varying appearances. By utilizing an equivariance model, it generates coherent videos of unseen motions from novel viewpoints, enhancing both flexibility and computational efficiency.