--- license: bsd-3-clause language: - en tags: - python - colour - color - colour-science - color-science - colour-spaces - color-spaces - colourspace - colorspace pipeline_tag: tabular-regression library_name: onnxruntime metrics: - mae model-index: - name: from_xyY (CIE xyY to Munsell) results: - task: type: tabular-regression name: CIE xyY to Munsell Specification dataset: name: CIE xyY to Munsell Specification type: munsell-renotation metrics: - type: delta-e value: 0.51 name: Delta-E CIE2000 - type: inference_time_ms value: 0.061 name: Inference Time (ms/sample) - name: to_xyY (Munsell to CIE xyY) results: - task: type: tabular-regression name: Munsell Specification to CIE xyY dataset: name: Munsell Specification to CIE xyY type: munsell-renotation metrics: - type: delta-e value: 0.48 name: Delta-E CIE2000 - type: inference_time_ms value: 0.066 name: Inference Time (ms/sample) --- # Learning Munsell - Machine Learning for Munsell Color Conversions A project implementing machine learning-based methods for bidirectional conversion between CIE xyY colourspace values and Munsell specifications. **Two Conversion Directions:** - **from_xyY**: CIE xyY to Munsell specification - **to_xyY**: Munsell specification to CIE xyY ## Project Overview ### Objective Provide 100-1000x speedup for batch Munsell conversions compared to colour-science routines while maintaining high accuracy. ### Results **from_xyY** (CIE xyY to Munsell) — 33 models evaluated on all 2,734 REAL Munsell colors: | Model | Delta-E | Speed (ms) | | ---------------------------------------------------------------------------------- | -------- | ---------- | | Colour Library (Baseline) | 0.00 | 116.3 | | **Multi-MLP + Multi-Error Predictor** | **0.51** | 0.061 | | Multi-ResNet + Multi-Error Predictor (Large Dataset) | 0.52 | 0.096 | | Transformer + Error Predictor (Large Dataset) | 0.52 | 0.163 | | Multi-MLP (Classification Code) + Code-Aware Multi-Error Predictor (Large Dataset) | 0.53 | 0.118 | | Multi-Head + Multi-Error Predictor (Large Dataset) | 0.53 | 0.046 | | Multi-MLP (Classification Code) + Code-Aware Multi-Error Predictor | 0.53 | 0.052 | | Multi-MLP (Classification Code) + Multi-Error Predictor | 0.53 | 0.050 | | MLP + Error Predictor | 0.53 | 0.036 | | Multi-Head + Multi-Error Predictor | 0.54 | 0.057 | | Multi-ResNet (Large Dataset) | 0.56 | 0.047 | | Multi-Head (Large Dataset) | 0.57 | 0.012 | | FT-Transformer | 0.70 | 0.067 | | Unified MLP | 0.71 | 0.074 | | Transformer (Large Dataset) | 0.73 | 0.120 | | Mixture of Experts | 0.74 | 0.021 | | Multi-MLP | 0.91 | 0.027 | | Multi-MLP (Classification Code) | 0.92 | 0.026 | | Multi-MLP (Classification Code) (Large Dataset) | 0.89 | 0.066 | | MLP + Self-Attention | 0.88 | 0.185 | | Deep + Wide | 1.18 | 0.078 | | MLP (Base Only) | 1.30 | **0.009** | | Multi-MLP (Hue Angle sin/cos) | 1.78 | 0.021 | - **Best Accuracy**: Multi-MLP + Multi-Error Predictor — Delta-E 0.51, 1,905x faster - **Fastest**: MLP Base Only (0.009 ms/sample) — 13,365x faster than Colour library - **Best Balance**: MLP + Error Predictor — 3,196x faster with Delta-E 0.53 **to_xyY** (Munsell to CIE xyY) — 6 models evaluated on all 2,734 REAL Munsell colors: | Model | Delta-E | Speed (ms) | | ------------------------------------- | -------- | ---------- | | Colour Library (Baseline) | 0.00 | 1.40 | | **Multi-MLP + Multi-Error Predictor** | **0.48** | 0.066 | | Multi-Head + Multi-Error Predictor | 0.51 | 0.054 | | Simple MLP | 0.52 | 0.002 | | Multi-MLP | 0.57 | 0.028 | | Multi-Head | 0.60 | 0.013 | | Multi-MLP + Error Predictor | 0.61 | 0.060 | - **Best Accuracy**: Multi-MLP + Multi-Error Predictor — Delta-E 0.48, 21x faster - **Fastest**: Simple MLP (0.002 ms/sample) — 669x faster than Colour library ### Approach - **33 models** tested for from_xyY (MLP, Multi-Head, Multi-MLP, Multi-ResNet, Transformers, FT-Transformer, Mixture of Experts) - **6 models** tested for to_xyY (Simple MLP, Multi-Head, Multi-MLP with error predictors) - **Two-stage models** (base + error predictor) proved most effective - **Best model**: Multi-MLP + Multi-Error Predictor with Delta-E 0.51 - **Training data**: ~1.4M samples from dense xyY grid with boundary refinement and forward Munsell sampling - **Deployment**: ONNX format with ONNX Runtime For detailed architecture comparisons, model benchmarks, training pipeline details, and experimental results, see [docs/learning_munsell.md](docs/learning_munsell.md). ## Installation **Dependencies (Runtime)**: - numpy >= 2.0 - onnxruntime >= 1.16 **Dependencies (Training)**: - torch >= 2.0 - scikit-learn >= 1.3 - matplotlib >= 3.9 - mlflow >= 2.10 - optuna >= 3.0 - colour-science >= 0.4.7 - click >= 8.0 - onnx >= 1.15 - onnxscript >= 0.5.6 - tqdm >= 4.66 - jax >= 0.4.20 - jaxlib >= 0.4.20 - flax >= 0.10.7 - optax >= 0.2.6 - scipy >= 1.12 - tensorboard >= 2.20 From the project root: ```bash cd learning-munsell # Install all dependencies (creates virtual environment automatically) uv sync ``` ## Usage ### Generate Training Data ```bash uv run python learning_munsell/data_generation/generate_training_data.py ``` **Note**: This step is computationally expensive (uses iterative algorithm for ground truth). ### Train Models **xyY to Munsell (from_xyY)** Best performing model (Multi-MLP + Multi-Error Predictor): ```bash # Train base Multi-MLP uv run python learning_munsell/training/from_xyY/train_multi_mlp.py # Train multi-error predictor uv run python learning_munsell/training/from_xyY/train_multi_mlp_multi_error_predictor.py ``` Alternative architectures: ```bash uv run python learning_munsell/training/from_xyY/train_multi_resnet_large.py uv run python learning_munsell/training/from_xyY/train_multi_head_large.py uv run python learning_munsell/training/from_xyY/train_ft_transformer.py uv run python learning_munsell/training/from_xyY/train_unified_mlp.py uv run python learning_munsell/training/from_xyY/train_deep_wide.py uv run python learning_munsell/training/from_xyY/train_mlp_attention.py ``` **Munsell to xyY (to_xyY)** Best performing model (Multi-MLP + Multi-Error Predictor): ```bash # Train base Multi-MLP uv run python learning_munsell/training/to_xyY/train_multi_mlp.py # Train multi-error predictor uv run python learning_munsell/training/to_xyY/train_multi_mlp_multi_error_predictor.py ``` Other architectures: ```bash uv run python learning_munsell/training/to_xyY/train_multi_head.py uv run python learning_munsell/training/to_xyY/train_multi_mlp_error_predictor.py uv run python learning_munsell/training/to_xyY/train_multi_head_multi_error_predictor.py ``` Train the differentiable approximator for use in Delta-E loss: ```bash uv run python learning_munsell/training/to_xyY/train_munsell_to_xyY_approximator.py ``` ### Hyperparameter Search ```bash uv run python learning_munsell/training/from_xyY/hyperparameter_search_multi_head.py uv run python learning_munsell/training/from_xyY/hyperparameter_search_multi_head_error_predictor.py ``` ### Compare All Models ```bash uv run python learning_munsell/comparison/from_xyY/compare_all_models.py uv run python learning_munsell/comparison/to_xyY/compare_all_models.py ``` Generates comprehensive HTML reports at `reports/from_xyY/model_comparison.html` and `reports/to_xyY/model_comparison.html`. ### Monitor Training **MLflow**: ```bash uv run mlflow ui --backend-store-uri "sqlite:///mlruns.db" --port=5000 ``` Open in your browser. ## Directory Structure ``` learning-munsell/ +-- data/ # Training data | +-- training_data.npz # Generated training samples | +-- training_data_large.npz # Large dataset (~1.4M samples) | +-- training_data_params.json # Generation parameters | +-- training_data_large_params.json +-- models/ # Trained models (ONNX + PyTorch) | +-- from_xyY/ # xyY to Munsell models (38 ONNX models) | | +-- multi_mlp_multi_error_predictor.onnx # BEST | | +-- ... (additional model variants) | +-- to_xyY/ # Munsell to xyY models (6 ONNX models) | +-- multi_mlp_multi_error_predictor.onnx # BEST | +-- ... (additional model variants) +-- learning_munsell/ # Source code | +-- analysis/ # Analysis scripts | +-- comparison/ # Model comparison scripts | +-- data_generation/ # Data generation scripts | +-- interpolation/ # Classical interpolation methods | +-- losses/ # Loss functions (JAX Delta-E) | +-- models/ # Model architecture definitions | +-- training/ # Model training scripts | +-- utilities/ # Shared utilities +-- docs/ # Documentation +-- reports/ # HTML comparison reports +-- logs/ # Script output logs +-- mlruns.db # MLflow experiment tracking database ``` ## About **Learning Munsell** by Colour Developers Research project for the Colour library