We found a match
Your institution may have rights to this item. Sign in to continue.
- Title
Intuitive teleoperation with hand-tracking in VR: a study on master–slave system virtualization and 3D workspace visualization.
- Authors
Yu, Seungnam; Kim, Tae Yop; Park, Won Woo; Lee, Se Hun; Han, Jonghui
- Abstract
This study proposed an intuitive teleoperation approach utilizing the hand-tracking mode supported by commercialized virtual reality devices. Unlike previous studies that used handheld controllers connected to an HMD (head-mounted display) system, this approach directly synchronizes the virtual hand (user's hand recognized by the VR device), the 3D model of the telemanipulator, and the practical telemanipulator simultaneously. The entire system was developed by integrating the previously developed master–slave system with Unity, a cross-platform game engine. In this platform, the physical slave system was converted into a virtual 3D model, and numerical optimization techniques were applied to avoid the dependency on conventional URDF (unified robot description format) applications. To verify the feasibility of this approach, several experiments were conducted, including motion consistency between the user and the robot hand, and trajectory-following performance. The rest of this paper presents the approaches for visualization of the workspace in VR. First, the pros and cons of images acquired through RGB cameras were reviewed, and then the point cloud-based 3D reconstruction approach, which was the focus of this study, was introduced. Considered methods included visualizing point clouds with the Mesh Generation Method of Unity and mapping these clouds using BAD SLAM (Bundle Adjusted Direct Simultaneous Localization and Mapping) and Open3D. Furthermore, various camera placements were tested to balance visual clarity and task efficiency, suggesting future improvements in VR visual quality and manipulation control.
- Publication
International Journal of Advanced Manufacturing Technology, 2024, Vol 134, Issue 5/6, p2353
- ISSN
0268-3768
- Publication type
Article
- DOI
10.1007/s00170-024-14213-3