GESTURE RECOGNITION USING OPEN CV AND PYTHON

Hand gesture recognition can easily be done using python by detecting contours and convex hull of the palm region using simple image processing techniques in Open CV . The aim of the project was to devise a program that is able to detect out hands, track them in real-time and perform some gesture recognition.

The algorithm is given below,

CAPTURE THE IMAGE AND CONVERT IT INTO GRAY SCALE

Here the image of  our hand ,which is the region of interest , captured , which is then converted to gray-scale using  cvtColor()  function.2017-05-24 10_29_36-gesture all.png

cap = cv2.VideoCapture(0)
while(cap.isOpened()):
    ret, img = cap.read()
    grey = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

Here the captured image which is in BGR format is converted to

BLUR THE ROI

Here we are using Gaussian Blurring on the original image in order to smoothen  and to reduce noise and details from the image. We are not interested in the details of the image but in the shape of the object to track.By blurring, we create smooth transition from one color to another and reduce the edge content.

The function used here is Gaussianblur()

2017-05-24 12_57_56-sjpe.png

blurred = cv2.GaussianBlur(grey, (35,35), 0)

 

THRESHOLDING

Now thresholding of the image is done which replaces the pixels values to either black or white with respect to the threshold value.The function used here is threshold().

2017-05-24 10_30_00-Thresholded.png

thresh1 = cv2.threshold(blurred, 127, 255,cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)

Here 127 is the threshold value.The white area have a pixel value less than 127 and the black portion have a pixel value greater than 127.

DRAW CONTOURS

Contours can be explained simply as a curve joining all the continuous points (along the boundary), having same color or intensity. The contours are a useful tool for shape analysis and object detection and recognition.For better accuracy, use binary images. So before finding contours, apply threshold or canny edge detection.

The function used here is findContours().

2017-05-24 10_41_20-drawing

contours, hierarchy = cv2.findContours(thresh1.copy(),cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)

There are three arguments in cv2.findContours() function, first one is source image, second is contour retrieval mode, third is contour approximation method and its output is a modified image, the contours and hierarchy. Contours is a Python list of all the contours in the image.

FIND CONVEX HULL ANS CONVEXITY DEFECTS

Here, cv2.convexHull() function checks a curve for convexity defects and corrects it. Generally speaking, convex curves are the curves which are always bulged out, or at-least flat. And if it is bulged inside, it is called convexity defects. For example, check the below image of hand. Red line shows the convex hull of hand. The double-sided arrow marks shows the convexity defects, which are the local maximum deviations of hull from contours.

2017-05-24 18_27_47-Contour Features — OpenCV-Python Tutorials 1 documentation.png

hull = cv2.convexHull(cnt)

2017-05-24 11_02_18-Gesture.png

The above figure shows the convexity defects.We can count the number of defects to find out the number of fingers extended.

2017-05-24 10_51_21-Gesture.png

To know more details about the project reach us on info@bforge.com

 

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s