Skip to content

eleboss/EBR

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

19 Commits
 
 
 
 
 
 

Repository files navigation

Neuromorphic Synergy for Video Binarization (Academic Use Only)

Usage

We release the matlab code for quick test and understanding. Open matlab and run the main.m I have already prepared the data to run, so you will see the results in subfolders.

Abstract

Bimodal objects, such as the checkerboard pattern used in camera calibration, markers for object tracking, and text on road signs, to name a few, are prevalent in our daily lives and serve as a visual form to embed information that can be easily recognized by vision systems. While binarization from intensity images is crucial for extracting the embedded information in the bimodal objects, few previous works consider the task of binarization of blurry images due to the relative motion between the vision sensor and the environment. The blurry images can result in a loss in the binarization quality and thus degrade the downstream applications where the vision system is in motion. Recently, neuromorphic cameras offer new capabilities for alleviating motion blur, but it is non-trivial to first deblur and then binarize the images in a real-time manner. In this work, we propose an event-based binary reconstruction method that leverages the prior knowledge of the bimodal target's properties to perform inference independently in both event space and image space and merge the results from both domains to generate a sharp binary image. We also develop an efficient integration method to propagate this binary image to high frame rate binary video. Finally, we develop a novel method to naturally fuse events and images for unsupervised threshold identification. The proposed method is evaluated in publicly available and our collected data sequence, and shows the proposed method can outperform the SOTA methods to generate high frame rate binary video in real-time on CPU-only devices.

Demo

Download model and data

In our paper, we conduct experiments on three types of data:

  • HQF contains synthetic blurry images and real-world events from HQF, where blurry images are generated using the same manner as GoPro.
  • Reblur contains real-world blurry images and real-world events from Reblur.
  • EBT is our collected dataset, it contains simulated event sequence using ESIM and real-world test data EBT.

EBT- Event-based Bimodal Target dataset

Examples of the real sequences of EBT dataset: Example of real sequence of EBT dataset

Examples of the synthetic sequences of EBT dataset: Example of real sequence of EBT dataset

About

Repo for event-based binary image reconstruction.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages