Hand Tracking and Virtual Mouse using Computer Vision

Track hand movements and create a virtual mouse using computer vision.

2 min readOct 23, 2021
Photo by Greg Rakozy on Unsplash


In previous articles, I went through with you the latest development in computer vision and optical character recognition using machine learning. In this article, let’s go through interesting stuff that we can build.

This Youtube channel is a good start for anyone who wants to learn the practical usage of computer vision.

The sample code I demonstrated here is adapted from the channel with the following improvements.

  • For GUI automation, instead of using autopy, I refactored the code to use PyAutoGUI.
  • Instead of an infinite loop, allow the user to type “q” to quit the application and then perform a proper resource clean-up.
  • Better exception handling when the movement is not detected.

Hand Tracking

Using the open-source Google MediaPipe library, we can track hand movements easily.




Software engineer, Data Science and ML practitioner.