US20260320889
2026-10-15
Physics
G06T13/40
The patent application describes a system that integrates robotics and virtual reality to assist users in physical interactions. This system employs an end effector, which moves in the real world, and a corresponding avatar that appears in a virtual space. The avatar's movements are synchronized with the end effector, allowing users to perceive interactions with the avatar rather than the robot itself. Users can view this virtual space through a wearable display, enhancing their experience with the system.
Virtual environments like the metaverse allow user interaction but lack physical interaction capabilities. Devices such as head-mounted displays enhance immersion through visual and audio feedback, but they do not provide physical interaction. While haptic devices add a sense of touch, they are often cumbersome and application-specific. Augmented reality provides information but does not facilitate physical tasks. Tele-robotics allows remote robot operation, transmitting audio, visual, and haptic information to the operator, yet it lacks the ability to project a human-like presence.
The disclosed system includes a moveable end effector in the real world and a display for the user to view a virtual space. A processor captures the end effector's pose and generates an avatar that mimics its movement in virtual space. The system can be worn by the user, projecting a virtual human that appears to interact with them. The robot can be operated remotely, allowing a distant operator to engage with the user through the avatar. This setup provides a blend of physical and virtual interaction, enhancing user comfort and immersion.
One potential application is tele-nursing, where an elderly person or patient requires assistance from a remote caregiver. A robot, controlled remotely, interacts with the patient, but the patient perceives the caregiver's avatar through a wearable display. This approach reduces discomfort and fear associated with interacting with robots and provides a sense of human presence. The synchronization of the avatar's movements with the robot's actions ensures a seamless and realistic interaction experience.
The system's synchronization of robot and avatar movements offers enhanced immersion through coordinated audio-visual-haptic stimuli. This creates a more realistic and humanistic interaction, fostering trust between users and operators. The technology can be applied in various settings where remote physical assistance is needed, providing a more comfortable and engaging experience for users who might be apprehensive about interacting with robots directly.