PIXEL DEPTH DETERMINATION FOR OBJECT

Smart overview of the Invention

The patent application details a method for enhancing augmented reality (AR) experiences by determining pixel depth for objects in images. The approach involves processing an image containing a person, extracting a portion of the image, and using machine learning models to predict depth information. This technique allows AR elements to be applied more realistically to the person depicted in the image, without relying on depth sensors. This innovation aims to improve the integration of virtual objects with real-world environments, enhancing user experiences in applications like messaging and gaming.

Background

Augmented Reality (AR) modifies a virtual environment by superimposing virtual objects onto the real world, unlike Virtual Reality (VR), which fully immerses users in a virtual setting. AR is used in various applications, including video games and messaging systems. Traditional AR systems often require depth sensors to determine object positions within images, which adds to device cost and complexity. The disclosed method seeks to eliminate this dependency on depth sensors while maintaining accurate AR object placement.

Innovative Approach

The disclosed system uses machine learning to predict the depth of specific points and relative pixel depths within an image portion depicting a person. By generating a dense depth reconstruction from these predictions, AR elements can be applied accurately and realistically. This method enhances image quality by avoiding incorrect background replacement, especially when multiple objects are present or when the entire body needs recognition beyond just facial features.

Technical Advantages

The proposed technique reduces reliance on hardware like depth sensors, lowering system resource demands and costs. It improves AR effect realism by accurately estimating pixel depths without affecting backgrounds. This advancement provides a more seamless integration of AR effects into real-world environments, improving user satisfaction with electronic devices equipped with AR capabilities.

Network Environment

The system operates within a networked computing environment involving client devices and messaging servers. These devices can include smartphones, tablets, AR glasses, and more, capable of executing the described operations. Messaging clients communicate over networks to exchange data and perform functions related to AR content presentation. The server-side infrastructure supports this functionality by handling data processing and storage, ensuring efficient operation across various client devices.