• Link: https://pan.baidu.com/s/1-KVYVTu0oxA_RzzwrylmFg?pwd=5rke

Note: Due to the separate data generation process, the KAIST synthetic dataset is organized into two folders:

• evt_rec_h5_dataset: Contains simulated raw events, VIS and LWIR frames, and optical flows (calculated with mmflow).
• infrarred_deblur_h5_dataset_with_vis_gt_blur_newnew_thin_copy: Contains blurry LWIR frames and corresponding sharp frames.

Advice: The dataset is approximately 400GB in size. You can download a subset of files first to run the code.

We obtained the original KAIST dataset from:
https://github.com/SoonminHwang/rgbt-ped-detection/tree/master/data

The original dataset files can be downloaded from:

• http://multispectral.kaist.ac.kr/pedestrian/data-kaist/videos.tar
• http://multispectral.kaist.ac.kr/pedestrian/data-kaist/annotations.tar

Since these links are no longer available, we are also providing the original data files via Baidu Netdisk for generating synthetic datasets with other settings:

• Baidu Netdisk Link: https://pan.baidu.com/s/1GW-dXjXBaubee2-MrPtpsA?pwd=k38d

Update the paths in the dataset .txt files to match your local environment.

To train tasks 1-3, navigate to the corresponding task's folder and run:

bash run_train.sh

Make sure to check and modify the configuration files to set the correct paths for the dataset, save directories, and any pretrained models if needed. The pretrained model needed for task 1 (update_reconstruction_model.pth) can be downloaded from event_cnn_minimal.

For inference, run:

bash run_inference.sh

Some quick ablation study results (task 3 trained with run_train.sh, get fused frames with run_inference.sh, then tested with VIFB):

This code is based on event_cnn_minimal and EFNet, we also take inspiration from other works such as SeAFusion and YDTR, thanks to these great works.