Next Generation Computer Vision using Machine Learning

A starter guide to computer vision using MediaPipe and OpenCV.

Photo by Greg Rakozy on Unsplash

Overview

In this article, let’s explore computer vision and the latest development in this area. I am going to show you code snippets using MediaPipe by Google to demonstrate various ML solutions and explore what is possible with the library.

Codebase

Let’s establish the codebase to use for all scenarios. The following code snippet is used to either capture video from your device camera or a video file.

• Using OpenCV, I can capture video either from a device camera or a video file.

# cap = cv2.VideoCapture(0) # Capture from camera. Adjust the number
  cap = cv2.VideoCapture(capture) # Capture from video file,e.g. mp4
.....
if __name__ == "__main__":
    show_video("computer-vision/dancing.mp4")

• Frames per second are also calculated.

def calculate_fps(img, pTime, pos=(50, 80), color=(0, 255, 0), scale=4, thickness=4):
    cTime = time.time()
    fps = 1 / (cTime - pTime)
    pTime = cTime
    cv2.putText(
        img,
        f"FPS: {int(fps)}",
        pos,
        cv2.FONT_HERSHEY_PLAIN,
        scale,
        color,
        thickness,
    )
    return pTime, img

• To exit the application, press q.

# Wait for key to exit
if cv2.waitKey(10) & 0xFF == ord("q"):
   break

Hands

The hand is annotated with 21 3D coordinates, as shown below.