Pit One Against Many: Leveraging Attention-head Embeddings for Parameter-efficient Multi-head Attention
Paper • 2310.07911 • Published • 1
Huiyin Xue
klein9692
AI & ML interests
None yet
Recent Activity
authored a paper about 5 hours ago
Pit One Against Many: Leveraging Attention-head Embeddings for
Parameter-efficient Multi-head Attention authored a paper about 5 hours ago
Deconstructing Attention: Investigating Design Principles for Effective
Language Modeling authored a paper about 5 hours ago
HashFormers: Towards Vocabulary-independent Pre-trained TransformersOrganizations
None yet
authored 4
papers about 5 hours ago
Deconstructing Attention: Investigating Design Principles for Effective Language Modeling
Paper • 2510.11602 • Published • 15
HashFormers: Towards Vocabulary-independent Pre-trained Transformers
Paper • 2210.07904 • Published
MultiHashFormer: Hash-based Generative Language Models
Paper • 2606.28057 • Published • 16
updated 16
models about 13 hours ago
klein9692/hf_qwen3_100m-model_10624_3_0_64_twe
Updated • 4
klein9692/hf_qwen3_100m-model_16384_4_0_64_twe
Updated
klein9692/hf_qwen3_100m-model_ori
Updated
klein9692/hf_qwen3_100m-model_add4L
Updated • 2 • 1
klein9692/mhf_1b_32768_4_64
Updated • 73 • 1
klein9692/mhf_1b_8192_2_64
Updated
klein9692/shf_1b_4096_4_0_64
Updated • 36
klein9692/shf_1b_8192_4_0_64
Updated • 37
klein9692/shf_1b_16384_4_0_64
Updated • 108
klein9692/mhf_1b_16384_3_64
Updated • 109
klein9692/mhf_1b_4096_4_64
Updated • 107
klein9692/mhf_1b_8192_4_64
Updated • 107
klein9692/mhf_1b_8192_3_64
Updated • 107
klein9692/mhf_1b_16384_4_1_64
Updated • 38
klein9692/mhf_1b_16384_4_4_64
Updated • 37
klein9692/mhf_1b_16384_4_3_64
Updated • 36