Qwen3-Coder-Next-REAP-48B-A3B-oQ3

This model was quantized using oQ mixed-precision quantization.

Quantization details

• Model type: qwen3_next
• Bits: 3
• Group size: 64
• Format: MLX safetensors

Benchmark

Model	File size	MMLU	JMMLU	HELLASWAG	GSM8K	ARC_CHALLENGE
Qwen3-Coder-Next-REAP-48B-A3B-oQ3	20.86 GB	64.3%	56.0%	80.3%	94.3%	85.7%
Qwen3-Coder-Next-REAP-48B-A3B-oQ3.5	22.41 GB	65.0%	57.7%	83.3%	95.0%	84.3%
Qwen3-Coder-Next-REAP-48B-A3B-oQ4	26.20 GB	62.0%	57.3%	81.0%	95.3%	85.7%
Qwen3-Coder-Next-REAP-48B-A3B-mlx-mxfp4	24.20 GB	62.7%	54.7%	76.0%	93.3%	85.7%
Qwen3-Coder-Next-REAP-q4-mlx	35.18 GB	66.3%	62.0%	78.0%	94.3%	84.0%

Detail

Model	Benchmark	Accuracy	Correct	Total	Time(s)
Qwen3-Coder-Next-REAP-48B-A3B-mlx-mxfp4	HELLASWAG	76.0%	228	300	293.8
Qwen3-Coder-Next-REAP-48B-A3B-mlx-mxfp4	ARC_CHALLENGE	85.7%	257	300	180.9
Qwen3-Coder-Next-REAP-48B-A3B-mlx-mxfp4	GSM8K	93.3%	280	300	1339.4
Qwen3-Coder-Next-REAP-48B-A3B-mlx-mxfp4	HELLASWAG	76.0%	228	300	293.8
Qwen3-Coder-Next-REAP-48B-A3B-mlx-mxfp4	ARC_CHALLENGE	85.7%	257	300	180.9
Qwen3-Coder-Next-REAP-48B-A3B-mlx-mxfp4	GSM8K	93.3%	280	300	1339.4
Qwen3-Coder-Next-REAP-48B-A3B-mlx-mxfp4	MMLU	62.7%	188	300	613.2
Qwen3-Coder-Next-REAP-48B-A3B-mlx-mxfp4	JMMLU	54.7%	164	300	225.9
Qwen3-Coder-Next-REAP-48B-A3B-oQ3	MMLU	64.3%	193	300	671.1
Qwen3-Coder-Next-REAP-48B-A3B-oQ3	JMMLU	56.0%	168	300	240.4
Qwen3-Coder-Next-REAP-48B-A3B-oQ3	HELLASWAG	80.3%	241	300	288.5
Qwen3-Coder-Next-REAP-48B-A3B-oQ3	ARC_CHALLENGE	85.7%	257	300	182.7
Qwen3-Coder-Next-REAP-48B-A3B-oQ3	GSM8K	94.3%	283	300	1261.6
Qwen3-Coder-Next-REAP-48B-A3B-oQ3.5	MMLU	65.0%	195	300	711.8
Qwen3-Coder-Next-REAP-48B-A3B-oQ3.5	JMMLU	57.7%	173	300	252.7
Qwen3-Coder-Next-REAP-48B-A3B-oQ3.5	HELLASWAG	83.3%	250	300	325
Qwen3-Coder-Next-REAP-48B-A3B-oQ3.5	ARC_CHALLENGE	84.3%	253	300	179.8
Qwen3-Coder-Next-REAP-48B-A3B-oQ3.5	GSM8K	95.0%	285	300	1377.4
Qwen3-Coder-Next-REAP-48B-A3B-oQ4	MMLU	62.0%	186	300	703.4
Qwen3-Coder-Next-REAP-48B-A3B-oQ4	JMMLU	57.3%	172	300	245.7
Qwen3-Coder-Next-REAP-48B-A3B-oQ4	HELLASWAG	81.0%	243	300	307.2
Qwen3-Coder-Next-REAP-48B-A3B-oQ4	ARC_CHALLENGE	85.7%	257	300	180.4
Qwen3-Coder-Next-REAP-48B-A3B-oQ4	GSM8K	95.3%	286	300	1411.5
Qwen3-Coder-Next-REAP-q4-mlx	MMLU	66.3%	199	300	858.2
Qwen3-Coder-Next-REAP-q4-mlx	JMMLU	62.0%	186	300	278.9
Qwen3-Coder-Next-REAP-q4-mlx	HELLASWAG	78.0%	234	300	377
Qwen3-Coder-Next-REAP-q4-mlx	ARC_CHALLENGE	84.0%	252	300	181.7
Qwen3-Coder-Next-REAP-q4-mlx	GSM8K	94.3%	283	300	1516.3