Tracking the blob of light from a flashlight can be useful. It certainly was for my Google Science fair project, and it may also be useful for any projects of your own. So without further ado, here’s the flashlight blob tracker.

Prerequisites

You should be able to understand the code in my previous post and should also have a strong foundation in python.

Create new thresholded frame

So far, we’ve been using the cv2.cvtColor() function simply in order to convert from an BGR colorspace to an HSV colorspace. But HSV colorspaces are only useful for thresholding specific colors or color ranges. This won’t work for thresholding a certain brightness. We’ll need a different colorspace for this.

#import libs
import cv2
import numpy as np
import time

#begin streaming
cap = cv2.VideoCapture(0)
while True:
    _, frame = cap.read()

    #convert frame to monochrome and blur
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    blur = cv2.GaussianBlur(gray, (9,9), 0)

    #use function to identify threshold intensities and locations
    (minVal, maxVal, minLoc, maxLoc) = cv2.minMaxLoc(blur)

    #threshold the blurred frame accordingly
    hi, threshold = cv2.threshold(blur, maxVal-20, 230, cv2.THRESH_BINARY)
    thr = threshold.copy()

    #resize frame for ease
    cv2.resize(thr, (300,300))

cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY). The unfamiliar parameter should be self explanatory, cv2.COLOR_BGR2GRAY, converts a frame from the BGR colorspace to the ‘gray’ or ‘monochrome’ colorspace. The reason this is done is intuitive. In a black and white image, the brightest area generally also appears to be the most white area in the picture. (The sun is the whitest area in this picture, and we know that it is also the brightest area in the picture).

cv2.minmaxloc() is also fairly self-explanatory. It searches the frame and returns the brightest pixel, the darkest pixel, and their respective positions. These values are then used as thresholds in threshold  frame. Our thresholded frame is called thr and threshold.

Identify light blob in thresholded frame

    #find contours in thresholded frame
    edged = cv2.Canny(threshold, 50, 150)
    lightcontours, hierarchy = cv2.findContours(edged, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)

    #attempts finding the circle created by the torch illumination on the wall
    circles = cv2.HoughCircles(threshold, cv2.cv.CV_HOUGH_GRADIENT, 1.0, 20,
                               param1=10,
                               param2= 15,
                               minRadius=20,
                               maxRadius=100,)

We use the cv2.HoughCircles function simply because one can intuitively confirm that a blob of light from a flashlight on a wall resembles a circular figure. NOTE: YOU MUST MESS AROUND WITH THE PARAMETERS ON THE HOUGH CIRCLES FUNCTION UNTIL  FALSE DETECTIONS ARE MINIMIZED

Track the light blob

All we have left is to make sure the blob detected matches the type we’re tracking for and then draw a marker around this light blob.

    #check if the list of contours is greater than 0 and if any circles are detected
    if len(lightcontours)>0 and circles is not None:
        #Find the Maxmimum Contour, this is assumed to be the light beam
        maxcontour = max(lightcontours, key=cv2.contourArea)
#avoids random spots of brightness by making sure the contour is reasonably sized
        if cv2.contourArea(maxcontour) > 2000:
            (x, final_y), radius = cv2.minEnclosingCircle(maxcontour)
            cv2.circle(frame, (int(x), int(final_y)), int(radius), (0, 255, 0), 4)
            cv2.rectangle(frame, (int(x) - 5, int(final_y) - 5), (int(x) + 5, int(final_y) + 5), (0, 128, 255), -1)
#display frames and exit
    cv2.imshow('light', thr)
    cv2.imshow('frame', frame)
    cv2.waitKey(4)
    key = cv2.waitKey(5) & 0xFF
    if key == ord('q'):
        break
cap.release()
cv2.destroyAllWindows()

Line 2 is to avoid an error that will appear if no circular light blobs are detected. If none are detected, the code just passes this iteration.

Line 4 assumes the light blob is the largest sized circular blob in the entire frame, meaning the maxcontour would be the blob itself.

Lines 6 is to make sure any tiny blobs or random bright pixels due to glare or other environmental interferences are not detected. The area must be over a certain value to be considered legit.

Lines 7-18  Just draw the trackers and display the frame.

After building a portion of your foundation, it’s best to continue learning by building something that’s not only useful, but also looks insanely impressive. So here you have it, a basic angle calculator.

UPDATE: WordPress is changing some of my code blocks to ‘amp’ and I haven’t yet found a way to fix this. For further guidance (although it would be a good exercise to infer), head over to my github repository.

LINK TO GITHUB GIST WITH ANGLECALC: https://gist.github.com/botforge/c88b842cafaa077a91048d51c2db0bdf

Prerequisites

This tutorial assumes you have some degree of proficiency with Python and can reasonably understand the OpenCV code here.

Determine HSV Range (again)

Before you continue writing the code you’ll need to use this HSV Trackbar to determine the Hue Low/High, Saturation Low/High and Value Low/High for the object you want to track. Mess around with the trackbars until you can only see the color of the object you are looking for. Repeat this process twice, for 2 differently colored objects. Note these values down, you will need them for later.

Filter for HSV Colors

Creating functions makes life a billion times easier, and allows you to organize your code much more effectively. I wrote the code initially with the functions findorange and findblue , although I eventually ended up using green and orange.

#import libs
import cv2
import numpy as np
import math

#uses distance formula to calculate distance
def distance((x1, y1), (x2,y2)):
    dist = math.sqrt((math.fabs(x2-x1))**2+((math.fabs(y2-y1)))**2)
    return dist

#filters for blue color and returns blue color position.
def findblue(frame):
    maxcontour = None
    blue = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
    bluelow = np.array([55, 74, 0])#replace with your HSV Values
    bluehi = np.array([74, 227, 255])#replace with your HSV Values
    mask = cv2.inRange(blue, bluelow, bluehi)
    res = cv2.bitwise_and(frame, frame, mask=mask)

#filters for orange color and returns orange color position.
def findorange(frame):
    maxcontour = None
    orange = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
    orangelow =  np.array([0, 142, 107])#replace with your HSV Values
    orangehi = np.array([39, 255, 255])#replace with your HSV Values
    mask = cv2.inRange(orange, orangelow, orangehi)
    res = cv2.bitwise_and(frame, frame, mask=mask)

Just remember to change the bluelow and bluehi and orangelow and orangehi array’s elements to those that suit your color choice. All of the functions used should be familiar from my tutorial on ‘Object Tracking and Following with OpenCV Python‘; read that if you don’t get some of it. What we’ve essentially done is sent the initial frame as a parameter to each of these functions, where they then convert to HSV and threshold for the color.

Return object positions

Next, you want to continue building on findblue() and findorange() by allowing them to return the coordinates of your objects.

def findblue(frame):
    blue = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
    bluelow = np.array([55, 74, 0])#replace with your HSV Values
    bluehi = np.array([74, 227, 255])#replace with your HSV Values
    mask = cv2.inRange(blue, bluelow, bluehi)
    res = cv2.bitwise_and(frame, frame, mask=mask)
    cnts, hir = cv2.findContours(mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
    if len(cnts) >0:
        maxcontour = max(cnts, key = cv2.contourArea)

        #All this stuff about moments and M['m10'] etc.. are just to return center coordinates
        M = cv2.moments(maxcontour)
        if M['m00'] > 0 and cv2.contourArea(maxcontour)>2000:
            cx = int(M['m10'] / M['m00'])
            cy = int(M['m01'] / M['m00'])
            return (cx, cy), True
        else:
            #(700,700), arbitrary random values that will conveniently not be displayed on screen
            return (700,700), False
    else:
        return (700,700), False

#filters for orange color and returns orange color position.
def findorange(frame):
    orange = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
    orangelow =  np.array([0, 142, 107])#replace with your HSV Values
    orangehi = np.array([39, 255, 255])#replace with your HSV Values
    mask = cv2.inRange(orange, orangelow, orangehi)
    res = cv2.bitwise_and(frame, frame, mask=mask)
    cnts, hir = cv2.findContours(mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
    if len(cnts) >0:
        maxcontour = max(cnts, key = cv2.contourArea)
        M = cv2.moments(maxcontour)
        if M['m00'] > 0 and cv2.contourArea(maxcontour)>2000:
            cx = int(M['m10'] / M['m00'])
            cy = int(M['m01'] / M['m00'])
            return (cx, cy), True
        else:
            return (700,700), False
    else:
        return (700,700), False

The cv2.Moments function and the cx and cy variable declarations are best explained in OpenCV’s introduction to contours. But simply put, lines n – n just return the coordinates of the center of the contour.

The reason we have the blogic boolean variable is just to validate that the object is present on screen. If the object isn’t present, this variable will be set to False, and the coordinates will be (700,700). I chose this set of points arbitrarily, as these, even when plotted, couldn’t be seen on my 300 x 400 window.

Distance Function

We’ll be using trigonometry to calculate the angle, so for this you’ll need to create a function that measures the distance between both points. For this, we use the standard distance equation  you should’ve learnt in high school.

#uses distance formula to calculate distance
def distance((x1, y1), (x2,y2)):
    dist = math.sqrt((math.fabs(x2-x1))**2+((math.fabs(y2-y1)))**2)
    return dist

Main Loop

#capture video
cap = cv2.VideoCapture(0)

while(1):
    _, frame = cap.read()
#if you're sending the whole frame as a parameter,easier to debug if you send a copy
    fra = frame.copy() 

     #get coordinates of each object
    (bluex, bluey), blogic = findblue(fra)
    (orangex, orangey), ologic = findorange(fra)
    #draw two circles around the objects (you can change the numbers as you like)
    cv2.circle(frame, (bluex, bluey), 20, (255, 0, 0), -1)
    cv2.circle(frame, (orangex, orangey), 20, (0, 128, 255), -1)

Our foundation is set. We’ve made a program that tracks the position of 2 different colored objects on screen, next, we need to apply trig to calculate the angle and display the entire setup in the most grandiose manner possible.

    if blogic and ologic:
        #quantifies the hypotenuse of the triangle
        hypotenuse =  distance((bluex,bluey), (orangex, orangey))
        #quantifies the horizontal of the triangle
        horizontal = distance((bluex, bluey), (orangex, bluey))
        #makes the third-line of the triangle
        thirdline = distance((orangex, orangey), (orangex, bluey))
        #calculates the angle using trigonometry
        angle = np.arcsin((thirdline/hypotenuse))* 180/math.pi

        #draws all 3 lines
        cv2.line(frame, (bluex, bluey), (orangex, orangey), (0, 0, 255), 2)
        cv2.line(frame, (bluex, bluey), (orangex, bluey), (0, 0, 255), 2)
        cv2.line(frame, (orangex,orangey), (orangex, bluey), (0,0,255), 2)

Our code is officially complete…sort of. If you run it, it’ll look great and work great but you’ll notice that it won’t detect any angle over 90 degrees. If you’re familiar with trig, you’ll know why, but to evade this situation and allow it to calculate angles until 180 degrees, we need a few more lines of code.

     #Allows for calculation until 180 degrees instead of 90
if orangey < bluey and orangex > bluex:
    cv2.putText(frame, str(int(angle)), (bluex-30, bluey), cv2.FONT_HERSHEY_SCRIPT_COMPLEX, 1, (0,128,220), 2)
elif orangey < bluey and orangex < bluex:
    cv2.putText(frame, str(int(180 - angle)),(bluex-30, bluey), cv2.FONT_HERSHEY_SCRIPT_COMPLEX, 1, (0,128,220), 2)
elif orangey > bluey and orangex < bluex:
    cv2.putText(frame, str(int(180 + angle)),(bluex-30, bluey), cv2.FONT_HERSHEY_SCRIPT_COMPLEX, 1, (0,128,220), 2)
elif orangey > bluey and orangex > bluex:
    cv2.putText(frame, str(int(360 - angle)),(bluex-30, bluey), cv2.FONT_HERSHEY_SCRIPT_COMPLEX, 1, (0,128, 229), 2)
if k == ord('q'): break

And that’s it! If the object tracker didn’t impress already, now you have a live angle calculator using just your camera.

Object tracking and the concepts learnt from developing an object tracking algorithm are necessary for computer vision implementation in robotics. By the end of this tutorial, you will have learnt to accurately track an object across the screen.

UPDATE: WordPress is changing some of my code blocks to ‘amp’ and I haven’t yet found a way to fix this. For further guidance (although it would be a good exercise to infer), head over to my github repository.

Prerequisites

This tutorial assumes you have some degree of proficiency with Python and can reasonably understand the OpenCV code here.

Determine HSV Range

Before you continue writing the code you’ll need to use this HSV Trackbar to determine the Hue Low/High, Saturation Low/High and Value Low/High for the object you want to track. Mess around with the trackbars until you can only see the color of the object you are looking for. Note these values down, you will need them for later.

Filter for HSV Color

#import necessary libraries
import cv2
import numpy as np
import time
#initialize the video stream
cap = cv2.VideoCapture(0)

#make two arrays, one for the points, and another for the timings
points = []
timer = []
while True:
    #start the timing
    startime = time.time()

    #append the start time to the array named 'timer'
    timer.append(g)

    #you only want to use the start time, so delete any other elements in the array
    del timer[1:]
    _, frame = cap.read()

    #resize and blur the frame (improves performance)
    sized = cv2.resize(frame, (600, 600))
    frame = cv2.GaussianBlur(sized, (7, 7), 0)

    #convert the frame to HSV and mask it
    hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
#fill in the values you obtained previously over here
    hlow = 17
    slow = 150
    vlow = 24
    hhigh = 78
    shigh = 255
    vhigh = 255
    HSVLOW  = np.array([hlow, slow, vlow])
    HSVHIGH = np.array([hhigh, shigh, vhigh])
    mask = cv2.inRange(hsv,HSVLOW, HSVHIGH)
    res = cv2.bitwise_and(frame,frame, mask =mask)

All of this stuff should be pretty straightforward after a few read-through’s. The only new function here is cv2.resize() and that itself is quite self explanatory (it resizes the frame). At this point, we have our new, ‘thresholded’ frame.

As a word of advice, make sure there isn’t a huge concentration of the color you’re looking for on the screen. For this basic object tracker, we’re only relying on color so if you have a lot of the color you want to track in the background, your best bet is to find a different colored object.

Find Maximum Contour

A lot of the time, you can simply visualize the algorithm necessary for solving most computer vision problem for robots if you understand what contours are. Since it is such a powerful tool, I suggest you build your foundation at this link (do the exercises, don’t just read), and come back when you kind of understand what contours are.

Once your done, try understanding this code.

    #create an edged frame of the thresholded frame
    edged = cv2.Canny(res, 50, 150)

    #find contours in the edged frame and append to the 'cnts' array
    cnts = cv2.findContours(edged, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)[0]

    # if contours are present
    if len(cnts)> 0:

        #find the largest contour according to their enclosed area
        c = max(cnts, key=cv2.contourArea)

        #get the center and radius values of the circle enclosing the contour
        (x, y), radius = cv2.minEnclosingCircle(c)

We started with the cv2.Canny()  function (documentation) with a min and max  threshold. Ignore the technicalities of the numbers, that essentially finds all of the edges in the frame.

Since cv2.findContours() returns a list, we need to find the largest contour (which we assume is our object) in this list. We use the max(cnts, key=cv2.contourArea). Thus, this function finds the area of all of the contours in the list, and then returns it’s maximum.

Following that we use the cv2.minEnclosingCircle(c) function to find the (x,y) coordinates of the center of the circle, and it’s radius.

At this point, we have tracked our object in the frame. All that’s left is to draw the circle and the trailing line. 

        centercircle = (int(x), int(y))
        radius = int(radius)
        cv2.circle(sized, centercircle, radius, (255, 30,255), 2) #this circle is the object
        cv2.circle(sized, centercircle, 5, (0, 0, 255), -1) #this circle is the moving red dot
        points.append(centercircle) #append this dot to the 'points' list
        if points is not None:
            for centers in points:
                cv2.circle(sized, centers, 5, (0, 0, 255), -1) #make a dot for each of the points

    #show all the frames and cleanup
    cv2.imshow('frame', sized)
    cv2.imshow('mask', res)
    k = cv2.waitKey(5) & 0xFF
    g = time.time()
    timer.append(g)

    #if 10 seconds have passed, erase all the points
    delta_t = timer[1] - timer[0]
    if delta_t >= 10:
        del timer[:]
        del points[:]
    if k == 27:
        break

Lines 1-9 essentially draw the circle around the object, and draw another small red circle at it’s center. This dot constitutes a point in the trail.  This point is then appended to the existing array ‘points’.

The for loop that follows cycles through each of the centers’ (x,y) coordinates (from the ‘points’ array) and draws another red dot in each position, effectively creating the trail.

Finally, another value is appended to the timer array. Delta_t computes the difference between the start and final times. If this value is greater than 10, all points are erased and a new trail is begun.