LittleBigAI

• 

  EmberNet CNN structure based on LeNet

• 

• 

• 

• 

• 

• 

• 

• 

  horse successfully detected

What it does

Self-driving vehicles sometimes suddenly brake for small objects, animals, and even shadows, putting lives of passengers in danger.

How we built it

Proposed solution is to build a CNN based on LeNet and train a ML model on CIFAR-10 dataset: {airplane, automobile, bird, cat, deer, dog, frog, horse, ship, truck}. When collision is imminent, use model to detect the type of object and decide whether to override braking based on the size of the object. CIFAR-10 used as a proxy dataset due to time constraints but can be modified for custom dataset to train for harmless vs dangerous objects. Algorithm will consider {bird, cat, dog, frog} to be small objects and {airplane, automobile, deer, horse, ship, truck} to be large objects.

EmberNet structure:

EmberNet implemented using PyTorch, CNN based on LeNet: cross-entropy loss function, SGD optimizer, lr = 0.01, batch size = 32. Test accuracy hovered around 50% after training for 100 epochs with 10000 images. Added dropout layers to fully connected section to fix overfitting, changed linear flattening layers to ReLU and added dropout layers. Training for 50 epochs with 50000 images resulted in ~60% test accuracy.

EmberNet exported to ONNX file, loaded on RPi. RPi camera capture using OpenCV, Haar Cascade face detection implemented as test.

Accomplishments that we're proud of

Implemented a custom made convolutional neural network using PyTorch and trained/tested on CIFAR-10 dataset.

Challenges we ran into

Scanning each frame for big objects results in model hallucinating trucks everywhere. Using higher confidence threshold results in no objects being detected, very low frame rate when attempting to scan with multiple different sized blocks.

What we learned

Changing linear flattening layers to ReLU function and adding dropout layer greatly reduces overfitting problem but does not solve it completely. OpenCV had issue with ONNX model due to unusual kernel sizes, used ONNX runtime instead:

import onnxruntime as ort

ort_session = ort.InferenceSession("EmberNet.onnx")

What's next for LittleBigAI

Use YOLO library and use CNN in conjunction to verify object detection.

Built With

• cifar-10
• cnn
• computer-vision
• opencv
• pytorch
• raspberry-pi