CROSS-DEVICE COMMUNICATION WITH ADAPTIVE AVATAR INTERACTION

Smart overview of the Invention

The patent application introduces a method and system for enabling cross-device communication with adaptive 3D avatar interaction. This technology facilitates seamless communication across devices with varying capabilities, such as VR headsets, AR devices, mobile phones, and desktop computers. The method involves creating a 3D avatar that adapts to different device features, enhancing the user experience by utilizing device-specific functionalities.

Technical Implementation

A key aspect involves receiving communication inputs from a first device to determine facial landmarks and head orientation of the user. This data is used to generate an animated 3D avatar. The system adjusts a virtual camera position based on the user's head orientation to maintain an immersive experience. Additional inputs from a second device allow the avatar to be modified with enhanced features, which are then provided back to the second device.

Device-Specific Features

The method supports various viewing modes such as picture-in-picture (PIP), side-by-side, and cinematic views. When interacting with devices like VR headsets, the system can incorporate VR-specific inputs and outputs to enrich the avatar animation. Similarly, AR-specific functionalities enable rendering the avatar in real-world environments, enhancing the interactive experience for users on AR devices.

Adaptive Interaction

The system can dynamically switch between VR and AR modes upon request from the second device, adjusting the avatar's animation and virtual camera position accordingly. For devices with audio capabilities, it generates facial expressions and mouth movements based on audio inputs, ensuring realistic interactions. Additionally, it calculates shoulder positions relative to head orientation for depth perception in avatar animations.

Applications and Flexibility

This technology addresses compatibility challenges in modern communication systems by adapting experiences across different platforms. By leveraging device-specific features, it ensures that advanced functionalities of high-capability devices are utilized effectively, even when interacting with less-capable devices. The integration of various components and features allows for flexible configurations tailored to specific communication needs.