Your Mixture-of-Experts LLM Is Secretly an Embedding Model For Free
Paper • 2410.10814 • Published • 51
MoE
-
-
Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment
Paper • 2502.16894 • Published • 33 -
Ring-lite: Scalable Reasoning via C3PO-Stabilized Reinforcement Learning for LLMs
Paper • 2506.14731 • Published • 8 -
SlimMoE: Structured Compression of Large MoE Models via Expert Slimming and Distillation
Paper • 2506.18349 • Published • 13 -
Mixture of Global and Local Experts with Diffusion Transformer for Controllable Face Generation
Paper • 2509.00428 • Published • 19 -
The Illusion of Specialization: Unveiling the Domain-Invariant "Standing Committee" in Mixture-of-Experts Models
Paper • 2601.03425 • Published • 17 -
ConceptMoE: Adaptive Token-to-Concept Compression for Implicit Compute Allocation
Paper • 2601.21420 • Published • 42 -
On Token's Dilemma: Dynamic MoE with Drift-Aware Token Assignment for Continual Learning of Large Vision Language Models
Paper • 2603.27481 • Published • 35 -
Co-Evolving Policy Distillation
Paper • 2604.27083 • Published • 67 -
Post-Trained MoE Can Skip Half Experts via Self-Distillation
Paper • 2605.18643 • Published • 30 -
MobileMoE: Scaling On-Device Mixture of Experts
Paper • 2605.27358 • Published • 11