SYNCHRONIZED, INTERACTIVE AUGMENTED REALITY DISPLAYS FOR MULTIFUNCTION DEVICES

Smart overview of the Invention

A multifunction device with a touch-sensitive surface can capture live video of a real-world environment and identify objects within that video. An information layer is generated that relates to these objects, which can include user annotations made via the touch-sensitive surface. This information layer is then overlaid on the live video, creating an augmented display. The device can also use onboard sensors to detect motion and synchronize the video and information layer as the camera perspective changes. This integrated display can be shared with other devices through a communication link.

Technical Field and Background

The application is situated within the realm of augmented reality (AR) technology, which merges real-world views with computer-generated imagery. Despite significant interest, many AR systems are complex and costly, limiting their accessibility for average consumers. The described system aims to simplify AR technology, making it more accessible and user-friendly for everyday use.

Functionality and Features

The device can present a split-screen display, with one section showing the live video and its information layer, and the other displaying computer-generated imagery of the objects. Users can navigate these images using either on-screen controls or by physically moving the device. The information layer can be enriched with annotations, such as text, images, or links, providing detailed insights into the objects captured in the live video.

Object Recognition and Annotation

Objects within the live video are identified either manually or automatically using advanced recognition techniques. Annotations, which could include text or visual highlights, are displayed alongside these objects to convey additional information, such as specifications or part numbers. The system can leverage various local or network information sources to enrich the information layer, ensuring the user receives comprehensive data.

Synchronization and Sharing

The device uses onboard sensors to maintain alignment of the information layer as the video perspective changes, avoiding the need for repeated object recognition. This synchronization allows users to move the device freely while maintaining accurate annotations. The system also supports sharing of the synchronized live video and information layers with other devices via wired or wireless connections, facilitating collaborative or multi-device use.