Synchronized, Interactive Augmented Reality Displays for Multifunction Devices

Smart overview of the Invention

A multifunction device can integrate live video of a real-world environment with an information layer on a touch-sensitive surface. Objects within the live video are identified, and related information is overlaid as annotations, which users can interact with through touch. This combined display adapts to the device's motion, maintaining synchronization between the video and annotations using onboard sensors. The resulting augmented reality (AR) experience can be shared with other devices via communication links.

Technical Field and Background

The technology pertains to AR applications on multifunction devices, addressing limitations of current AR systems that are often complex and costly. By overlaying computer-generated imagery onto live views, users gain enhanced interaction with their environment. This approach aims to make AR accessible for everyday consumers by simplifying system complexity and cost.

Functional Summary

Devices capture live video and identify objects, generating information layers that may include user annotations. These layers are synchronized with video feeds using sensor data to adjust for device movement. A split-screen display may show both the augmented video and separate computer-generated imagery, with navigation controls for user interaction or physical device movement.

Detailed Examples of Use

The device can display live video of objects like a circuit board, identifying components such as chips and capacitors. Annotations provide detailed information like part numbers or manufacturer details. Object recognition techniques are employed to assist in identifying and overlaying information accurately. Users can trigger additional data through interactions like touching components or using tools like a magnifying glass for zooming.

Synchronization and Sharing Capabilities

Devices utilize onboard sensors for real-time alignment of annotations during perspective changes, eliminating delays from repetitive object recognition processes. This allows seamless maneuvering around objects while maintaining annotation accuracy. Moreover, synchronized displays across multiple devices enable shared AR experiences by transmitting live video and corresponding information layers over wireless or wired connections.