Vanos007/nsga2-image-fusion

NSGA-II Optimized Deep Learning Image Fusion (Pansharpening)

Multi-objective optimization of deep learning pansharpening models using NSGA-II (Non-dominated Sorting Genetic Algorithm II). The optimizer tunes CNN hyperparameters to achieve balanced results across competing image quality objectives.

🎯 What This Does

Traditional pansharpening fuses a low-resolution multispectral (MS) image with a high-resolution panchromatic (PAN) image to produce a high-resolution multispectral image. This project uses NSGA-II to find the optimal trade-offs between:

Objective	Measures	Goal
SAM (Spectral Angle Mapper)	Spectral fidelity	Minimize ↓
ERGAS	Normalized spectral-spatial error	Minimize ↓
SF (Spatial Frequency)	Spatial detail/sharpness	Maximize ↑

These objectives fundamentally conflict: improving spatial sharpness often degrades spectral fidelity and vice versa. NSGA-II finds the Pareto front — the set of solutions where no objective can be improved without worsening another.

🏗️ Architecture

┌─────────────────────────────────────────────────┐
│              NSGA-II Optimizer (pymoo)           │
│  Population: 20-30 individuals                  │
│  Each individual = [lr, λ_spec, λ_spat,         │
│                      n_filters, n_blocks]        │
├─────────────────────────────────────────────────┤
│         ↓ For each individual:                   │
│  ┌──────────────────────────────────────┐       │
│  │    Z-PNN Fusion Network (PyTorch)     │       │
│  │  Input: [MS_upsampled ∥ PAN]          │       │
│  │  Architecture: Conv → ResBlocks → Conv │       │
│  │  Loss: λ_spec·L1 + λ_spat·SpatCorr   │       │
│  └──────────────────────────────────────┘       │
│         ↓ Train for N epochs                     │
│  ┌──────────────────────────────────────┐       │
│  │    Evaluate: SAM, ERGAS, SF            │       │
│  │    → fitness = [SAM, ERGAS, -SF]       │       │
│  └──────────────────────────────────────┘       │
│         ↓ Return to NSGA-II                      │
│  Selection → Crossover → Mutation → Next Gen     │
└─────────────────────────────────────────────────┘
         ↓ After all generations
    Pareto Front + Visualizations

📁 Project Structure

nsga2_image_fusion/
├── __init__.py            # Package init
├── metrics.py             # 10+ image quality metrics
├── model.py               # Z-PNN fusion network + loss functions
├── data.py                # Data pipeline (synthetic, H5, TIFF)
├── train.py               # Training loop per individual
├── nsga2_optimizer.py     # NSGA-II wrapper (pymoo)
├── visualize.py           # Pareto front plots & analysis
run_nsga2_fusion.py        # Main entry point
requirements.txt           # Dependencies
sample_outputs/            # Demo run outputs

🚀 Quick Start

pip install -r requirements.txt
python run_nsga2_fusion.py --mode demo     # ~2 min CPU
python run_nsga2_fusion.py --mode test     # ~15 min CPU
python run_nsga2_fusion.py --mode full --device cuda  # hours, GPU

🔧 Decision Variables

Variable	Range	Description
lr	[1e-5, 1e-2]	Learning rate (log-scale)
lambda_spectral	[0.1, 5.0]	Spectral loss weight
lambda_spatial	[0.1, 5.0]	Spatial loss weight
n_filters	{16..128}	Conv filter count
n_blocks	{2..8}	Residual block count

📖 References

• Z-PNN: Ciotola et al. (2021) arXiv:2111.08334
• λ-PNN: Ciotola et al. (2023) arXiv:2307.14403
• NSGA-II: Deb et al. (2002)
• pymoo: Blank & Deb (2020)

License

MIT