AVRP-Robo-Scarecrow

GIF
Robot model needed for calculation inverse kinematics
Demonstration of our object detection
GIF
Demonstration of bot moving and detecting a target

Inspiration

One of our members used to work as a "Bird Control Officer," where he learned that seagull overpopulation at landfills had become a hazard for operator navigation and their health. With up to tens of thousands of seagulls at a location, the birds can cause limited visibility and carry diseases including drug resistant E. coli (1).

Birds are also a hazard to the aviation industry which spends upwards of $1.65 Billion per year on bird strike damage and delays (2).

For the agriculture industry, bird damage to fruit, seeds, and horticultural crops costs the industry on average $313 million per annum (3).

Lastly, humans are also a hazard to birds. In 2010 over 1,600 ducks were killed when they landed on a tailings pond (4). It is estimated that by 2030 wind power will potentially kill at least one million birds a year (5). It is our responsibility to ensure their safety.

Bird deterrence requires active constant intelligent management for sometimes up to 14 hours a day and offers a perfect opportunity for a fleet of robots to work tirelessly each day.

What it does

Our project uses a small portable computer, a robotic arm and a stereo camera to target birds within its field of vision, tracks their position, and targets them with a laser pointer as a form of deterrence.

How we built it

Computer Vision:

The computer vision task can be further split into 2 tasks, finding all birds in frame and then finding each bird's position in 3d. To achieve this we first pass the image through a computer vision system (YOLOv7) to detect if there are any birds in range and generate bounding boxes around all the birds. We then pass the location of these bounding boxes to a 3d object detection module to find the location of the birds relative to the camera. Finally, the detections are filtered to determine the order in which the birds should be targeted.

Hardware:

A simple two degree of freedom arm powered by two servo motors enables our deterrent method (ideally a laser, water cannon or small projectile launcher) to accurately target a bird at a distance. Birds perceive a green laser beam as a physical danger causing them to fly away (6).

Robot Description & Inverse Kinematics:

This system is given the location of the target bird centered around the left camera. We first convert the coordinates of the target bird to the reference frame of the laser turret. We then apply inverse kinematics equations to calculate the angles for the servos such that the laser points at the targeted bird.

Challenges we ran into

One of the main challenges was creating a representation of the turret that could be used to generate the inverse kinematics equations. Inverse kinematics assumes that the robot joints can move to the necessary position, but this is often not the case due to the limited movement of our robot. Furthermore, with the addition of a laser pointer, we introduce an arm that varies in length to the model that drastically increases the complexity of the problem.

As well we had to get a very accurate measurement of the change from the camera reference frame to the turret reference frame. Since our targets are relatively far away from the axes of rotation, small errors in the position of the turret can result in very large errors in the position of the laser at the distance to the bird.

Lastly we had to combine multiple computer vision algorithms to only target birds and to track them in 3d space. This required us to build a system that passed data between camera and detection algorithms which increased the complexity.

Accomplishments that we're proud of

Putting together all the pieces of this project was not an easy task, there are many dependent and complicated subsystems that are required. We are proud that all of the subsystems came together with ease in the final project.

What we learned

We learned how to apply inverse kinematics from an offset reference frame and how to combine multiple vision models to achieve an accurate tracking system in 3d.

What's next for Robo-Scarecrow

We would like to switch to a custom trained detection model. This would improve the accuracy and confidence of detections. Doing this would also allow us to track specific species of birds and potentially have different deterrent methods depending on species.

Additionally, a custom model would improve detection at long ranges. Also, a camera with a smaller field of view and higher resolution could move back and forth to scan an area and detect birds smaller and farther away. Expanding the system to a network of several devices would also improve the coverage area.

The current servos used have a limited range of motion that prevent the system from targeting all the space around it. Upgrading the motors so that the system has full range of motion along with the above modifications to the camera would allow the system full 360 range of motion.