SphereFlow: Missing Modality Imputation via Geometric Transport on Hypersphere
Abstract
SphereFlow projects modalities onto a shared hypersphere to enable short-range cross-modal imputation via flow matching with geometry-aware noise and decoder modifications.
Missing modality imputation aims to synthesize unobserved modalities from observed ones. Existing methods treat each modality as living on its own latent manifold, forcing the generator to learn a long, unstructured jump between them. We present SphereFlow, which takes a fundamentally different view: by projecting all modalities into a shared hyperspherical latent space via a frozen self-supervised encoder, cross-modal imputation reduces to short-range geometric transport between nearby points on the sphere. Realizing this idea with flow matching, however, exposes two geometric conflicts: the standard Gaussian noise source is catastrophically mismatched with the unit-norm data manifold, and the linear interpolation path departs the sphere surface into a region where the decoder has no training signal. We resolve both with simple, geometry-aware modifications: replacing the noise source with the observed-modality latent so that the velocity field learns only the cross-modal displacement, and training the decoder with a time-weighted loss that extends its domain to the near-surface region traversed by the flow. We prove that this data-as-source formulation reduces the worst-case off-manifold deviation from a quantity that grows with the latent dimension to a small, dimension-independent bound governed by inter-modality similarity. Experiments on BraTS multi-modal MRI imputation and CT-MRI translation demonstrate state-of-the-art generation fidelity across all settings, while downstream brain tumor segmentation with our imputed data closes roughly eighty percent of the gap to the full-modality oracle. Code is available at https://github.com/WTHH031230/SphereFlow.
Community
π SphereFlow (NeurIPS 2026): Missing modality imputation via geometric transport on the hypersphere.
Key idea: Instead of treating each modality as living on its own latent manifold, we project all modalities into a shared hyperspherical latent space using a frozen self-supervised encoder. Cross-modal imputation then becomes short-range transport between nearby points on the sphere, not a long jump between unrelated manifolds.
Highlights:
- π― Data-as-source flow matching: we replace the Gaussian noise source with the observed-modality latent, so the velocity field only needs to learn the cross-modal displacement.
- π Theory: this cuts the worst-case off-manifold deviation from a bound that grows with latent dimension to a small, dimension-independent bound set by inter-modality similarity.
- π§© Time-weighted decoder training extends the decoder to the near-surface region the flow passes through.
- π SOTA generation fidelity on BraTS multi-modal MRI imputation and CTβMRI translation. Downstream tumor segmentation with our imputed data closes ~80% of the gap to the full-modality oracle.
π Code: https://github.com/WTHH031230/SphereFlow
Happy to discuss β questions and feedback are very welcome!
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- MUNITE: Unified Multimodal Latent Inference for Any-to-Any Multimodal Generation (2026)
- GAE: Learning a Geometry-Native Latent Space for 3D-Consistent World Generation (2026)
- Preserving the Manifold: Frozen Expert Alignment for Label-Efficient SAR Scene Understanding (2026)
- MaDeL: Manifold-Decomposed Feature Losses for Generative Modeling (2026)
- Semantic Latent Alignment for Flow Matching with Invertible Bi-Lipschitz Adapters (2026)
- FuseAdapt: Parameter-Efficient Multimodal Fusion for Semantic Segmentation with Missing Modalities (2026)
- Closing the Modality Gap via Simplex-Constrained Representations (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2611.04414 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 1
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper