INTERACTIONS WITH 3D VIRTUAL OBJECTS USING POSES AND MULTIPLE-DOF CONTROLLERS

Smart overview of the Invention

A wearable system is designed to display virtual content in a three-dimensional (3D) space, enabling users to interact with both physical and virtual objects in their environment. It includes a display system that presents a 3D view, a user input device for receiving commands, and sensors to detect the user's pose. These components work together to facilitate user interactions based on contextual information, allowing for dynamic engagement with virtual objects.

User Interaction Methods

The system allows users to interact with objects through various methods, such as selecting and moving them using poses or by manipulating a user input device. For instance, when a user moves the input device, the corresponding virtual object will follow that movement. Additionally, a technique known as cone casting enables users to target virtual objects by adjusting the virtual cone's aperture based on their head movements and contextual factors present in their environment.

Dynamic User Interface Adjustments

To minimize user fatigue and enhance interaction efficiency, the wearable system can automatically adjust its interface operations based on contextual information. For example, when a user turns their head towards a densely populated area of virtual objects, the system may narrow the cone aperture to limit selectable objects. Conversely, if the user looks in an area with fewer objects, the aperture can widen to include more targets or reduce the necessary movement to select an object.

Movement Multipliers for Enhanced Interaction

The system can also utilize movement multipliers that translate the user's physical movements into greater virtual object movements. This feature allows users to move distant virtual objects without needing to make extensive physical movements. When interacting with nearby objects, however, the multiplier can be set to one, ensuring precise one-to-one control over object positioning.

Advanced 3D Display Capabilities

The wearable device can present various types of visual content, including still images and videos, in both 2D and 3D formats. To create an authentic sense of depth, the display must generate accommodative responses that correspond with the perceived virtual depths. By providing different images for various depth planes and allowing users to focus on these images accordingly, the system enhances depth perception and overall user experience in VR, AR, and MR environments.