We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
Control the Robot Arm through Vision-Based Human Hand Tracking.
- Authors
Le Hoai Phuong; Vo Duy Cong
- Abstract
In this paper, hand tracking based on computer vision is developed to control the movement of a SCARA robot arm. The robot arm will move according to the movement of the human hand. Instead of using buttons on the teach-pendant or a computer control program to move the robot arm, the robot can now be easily controlled and positioned quickly by the movement of the operator's hand. A SCARA robot arm with two rotation joints and one translation motion is constructed for the validation system. Two states of the hand are recognized for controlling the vacuum cup to grasp the products. Stepper motors drive the robot arm. Arduino Uno is used as the main controller for controlling the stepper motors. The hand-tracking is performed by using the MediaPipe Hands framework developed by Google. The coordinates of 21 hand landmarks are extracted for further processing. A program is written on a personal computer to process the image to get the position and state of the hand. This position is transformed into the rotation angles of the robot's joints. Then, the angles and state are sent to the Arduino board. The Arduino board creates pulse signals to rotate the stepper motors. The experimental results show that the robot's trajectory is close to the hand trajectory at a low speed.
- Subjects
ARDUINO, 14th century; ROBOT control systems; INDUSTRIAL robots; ROBOT hands; GOOGLE Inc.; STEPPING motors; ROBOT motion; AUTOMATION
- Publication
FME Transactions, 2024, Vol 52, Issue 1, p37
- ISSN
1451-2092
- Publication type
Article
- DOI
10.5937/fme2401037P