Hand Tracking and Virtual Mouse using Computer Vision
Track hand movements and create a virtual mouse using computer vision.
In previous articles, I went through with you the latest development in computer vision and optical character recognition using machine learning. In this article, let’s go through interesting stuff that we can build.
This Youtube channel is a good start for anyone who wants to learn the practical usage of computer vision.
The sample code I demonstrated here is adapted from the channel with the following improvements.
- For GUI automation, instead of using autopy, I refactored the code to use PyAutoGUI.
- Instead of an infinite loop, allow the user to type “q” to quit the application and then perform a proper resource clean-up.
- Better exception handling when the movement is not detected.
Using the open-source Google MediaPipe library, we can track hand movements easily.
- MediaPipe provides 21 landmarks or 3D coordinates we can use to track and check any particular motion easily, e.g. coordinate 4, 8, 12, 16, 20 are the fingertips.
- By checking the particular landmarks, and measuring the distance, we can detect a particular movement, e.g. fingers up.
Now let’s create a virtual mouse.