Post
6402
š¶ Released mule-torch ā an unofficial PyTorch port of MULE (SF-NFNet-F0), SiriusXM/Pandora's music-audio embedding model (McCallum et al., ISMIR 2022).
No retraining: I re-implemented the architecture in pure PyTorch and transferred the original TensorFlow weights, then checked it layer by layer against the genuine TF pipeline.
ā End-to-end clip-embedding cosine 0.9999999 vs the original
ā ONNX backbone parity < 1e-6
ā 62.35M params (paper: ~62.4M)
ā Batched, GPU-native, ONNX-exportable ā none of which the original
š¤ Weights: matteospanio/mule
š» Code: https://github.com/matteospanio/mule-torch
š¦ PyPI: https://pypi.org/project/mule-torch/
The fun bug: parity was perfect through every conv but the block output was anti-correlated (cos = ā1). Cause: the learnable skip-init gains couldn't be mapped by layer name (Keras scrambles the order) ā they had to be recovered from the graph.
ā ļø Unofficial, community port ā not affiliated with or endorsed by the original authors. All credit to them; please cite the paper. Weights inherit CC-BY-NC-4.0.
No retraining: I re-implemented the architecture in pure PyTorch and transferred the original TensorFlow weights, then checked it layer by layer against the genuine TF pipeline.
ā End-to-end clip-embedding cosine 0.9999999 vs the original
ā ONNX backbone parity < 1e-6
ā 62.35M params (paper: ~62.4M)
ā Batched, GPU-native, ONNX-exportable ā none of which the original
Analysis pipeline doespip install mule-torchfrom mule_torch import MuleModel
emb = MuleModel.from_pretrained()(waveform) # (B, T)@16kHz -> (B, 1728)š¤ Weights: matteospanio/mule
š» Code: https://github.com/matteospanio/mule-torch
š¦ PyPI: https://pypi.org/project/mule-torch/
The fun bug: parity was perfect through every conv but the block output was anti-correlated (cos = ā1). Cause: the learnable skip-init gains couldn't be mapped by layer name (Keras scrambles the order) ā they had to be recovered from the graph.
ā ļø Unofficial, community port ā not affiliated with or endorsed by the original authors. All credit to them; please cite the paper. Weights inherit CC-BY-NC-4.0.
