• [2026/01] 🔥 Our paper has been accepted by ICASSP 2026!

This repository contains the official PyTorch implementation of HSSDCT (Hierarchical Spatial-Spectral Dense Correlation Network).

HSSDCT introduces a novel framework for fusing low-resolution hyperspectral images (LR-HSI) with high-resolution multispectral images (HR-MSI). Unlike recent Transformer-based methods that suffer from quadratic complexity, our method proposes:

• Hierarchical Dense-Residue Transformer Block (HDRTB): Progressively enlarges receptive fields with dense-residue connections for multi-scale feature aggregation.
• Spatial-Spectral Correlation Layer (SSCL): Explicitly factorizes spatial and spectral dependencies, reducing self-attention to linear complexity while mitigating spectral redundancy.

Extensive experiments demonstrate that HSSDCT achieves state-of-the-art performance with significantly lower computational costs compared to recent methods like FusionMamba and QRCODE.

• Environment Setup
• Data Preparation
• Usage
  • Training
  • Testing / Inference
• Project Structure
• Configuration Options
• Evaluation Metrics
• Citation
• Acknowledgments

• Python 3.8 or higher
• CUDA 12.x compatible GPU (recommended: NVIDIA GPU with ≥16GB memory)
• Conda (recommended) or pip

# Clone the repository
git clone https://github.com/your-username/HSSDCT.git
cd HSSDCT

# Create a virtual environment (optional but recommended)
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Create conda environment
conda create -n hssdct python=3.10
conda activate hssdct

# Install PyTorch with CUDA support
conda install pytorch torchvision pytorch-cuda=12.1 -c pytorch -c nvidia

# Install remaining dependencies
pip install -r requirements.txt

The dataset should be organized in the following structure under your data root directory:

<data_root>/
├── train/
│   ├── <scene_name>/
│   │   ├── GT.npz.npy          # Ground truth HRHSI (256×256×172), float32
│   │   ├── hrmsi.npz           # High-resolution MSI, contains keys: 'hrmsi4', 'hrmsi6'
│   │   └── lrhsi.npz           # Low-resolution HSI, contains keys: 'lrhsi', 'lrhsi1', 'lrhsi2', 'lrhsi3'
│   ├── Hama9_15/
│   ├── TBD4_39/
│   └── ...
├── val/
│   ├── <scene_name>/
│   │   ├── GT.npz.npy
│   │   ├── hrmsi.npz
│   │   └── lrhsi.npz
│   └── ...
└── test/
    ├── <scene_name>/
    │   ├── GT.npz.npy
    │   ├── hrmsi.npz
    │   └── lrhsi.npz
    └── ...

To train the model from scratch:

python train.py \
    --root /path/to/your/data \
    --train_file ./data_path/train.txt \
    --val_file ./data_path/val.txt \
    --prefix EXPERIMENT_NAME \
    --batch_size 6 \
    --epochs 1000 \
    --lr 0.000055 \
    --lr_scheduler cosine \
    --msi_bands 4 \
    --bands 172 \
    --crop_size 128 \
    --image_size 256 \
    --network_mode 1 \
    --device cuda:0

python train.py \
    --resume_ind 100 \
    --resume_ckpt ./checkpoint/EXPERIMENT_NAME/best.pth \
    [other arguments...]

HSSDCT/
├── train.py              # Main training script (training loop & validation)
├── dataset.py            # Dataset classes for data loading
│                         # - Pairwise (LRHSI + HRMSI) & Triplet loading
├── trainOps.py           # Training utilities & Evaluation metrics
│                         # - Losses: SAM Loss, BandWise MSE
│                         # - Metrics: PSNR, ERGAS, RMSE
├── utils.py              # General utilities (Activation, padding)
├── requirements.txt      # Python dependencies
├── models/
│   ├── __init__.py
│   ├── hssdct.py         # Main model architecture
│   │                     # - HSSDCT Framework
│   │                     # - HDRTB (Hierarchical Dense-Residue Transformer Block)
│   │                     # - SSCL (Spatial-Spectral Correlation Layer)
│   └── module_util.py    # Weight initialization utilities
└── data_path/
    ├── train.txt         # Training sample list
    ├── val.txt           # Validation sample list
    └── test.txt          # Test sample list

The model is evaluated using the following metrics:

Quantitative comparison on the AVIRIS dataset (see Table 1 in the paper):

Visual comparison of fused results (Figure 5 from the paper).

If you find this work useful in your research, please consider citing:

@inproceedings{lee2026hssdct,
  title     = {HSSDCT: Factorized Spatial-Spectral Correlation for Hyperspectral Image Fusion},
  author    = {Lee, Chia-Ming and Ho, Yu-Hao and Lin, Yu-Fan and Lee, Jen-Wei and Kang, Li-Wei and Hsu, Chih-Chung},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  year      = {2026}
}

This project is licensed under the MIT License - see the LICENSE file for details.

For questions or issues, please open a GitHub issue or contact [jemmy112322@gmail.com].