System and Method for Markerless Motion Capture

Smart overview of the Invention

The system captures markerless motion data of a user using a camera, which is then applied to a 3D model of an avatar linked to a unique identity non-fungible token (NFT). This process animates the avatar in real-time, reflecting the user's movements. The avatar, represented by an NFT, is recognized across both Web2 and Web3 environments, ensuring a consistent identity for the user.

Motion Capture and Retargeting

The motion capture process involves identifying specific points on the user to track their movements. These points are divided into two groups based on their proximity to the camera. This data is then retargeted onto the 3D avatar model, allowing for accurate animation of the avatar's hands, face, and body in real-time.

Rendering and NFT Generation

Once the motion data is applied to the avatar, a video is rendered locally on the computing system. This video showcases the animated avatar and is associated with a newly generated NFT. This NFT serves as proof of ownership for the video, allowing users to control and potentially monetize their digital content.

System Components

The system comprises a processor and memory that execute instructions to perform operations such as capturing motion data and rendering videos. It includes components like user devices, server systems, blockchains, and NFT marketplaces connected through various network types. These networks may use secure connections for transmitting personal or confidential data.

Applications in the Metaverse

This technology supports user interaction within metaverse environments, which may include both centralized and decentralized platforms accessible via devices like mobile phones, computers, and XR devices. Users can utilize their NFT avatars in digital content shared across social media platforms, enhancing their presence and engagement in virtual spaces.