GESTURE-CONTROLLED VIRTUAL REALITY SYSTEMS AND METHODS OF CONTROLLING THE SAME

Smart overview of the Invention

The invention involves a gesture-controlled virtual reality (VR) system that integrates both on-body and off-body sensors to interpret user movements and gestures. These sensors collect signals related to the user's body part movements or positions relative to virtual objects. A processor then uses this data to generate gesture and position information, determining the user's intended actions within the VR environment and producing corresponding outputs.

Gesture Recognition

Gesture recognition is achieved by tracking body part movements, such as hand rotations or finger bends. Cameras can capture these movements for processing, but they often introduce latency and struggle with subtle motions. To address this, the system incorporates wearable sensors like gloves equipped with flex sensors and accelerometers, which provide granular motion data without the limitations of traditional cameras.

Position Detection

Position-detecting devices, including cameras, RF-based local triangulation systems, and wearable sensors, gather real-world physical position data of body parts. Ultra-wideband (UWB) radios are used for precise positioning by transmitting signals to receivers in the environment. This combination of technologies allows for detailed gesture recognition, enabling fine motor skill simulations like playing a virtual musical instrument.

Virtual Instrument Interaction

Users can interact with virtual musical instruments through gestures that mimic real-world playing techniques. The system captures hand movements and finger bends to determine intended musical notes. By mapping this data to virtual instruments, the system can generate real-time audio outputs. It also supports creating hybrid instruments by combining elements from different instruments, offering unique musical experiences.

Applications Beyond Music

While primarily illustrated with virtual musical instruments, the technology extends to various applications such as gaming, sports simulations, and medical rehabilitation. The system's ability to interpret detailed gestures makes it versatile for any scenario requiring intricate user interaction within a VR setting.