GLM-4.7-Flash-REAP-23B-A3B-oQ5

This model was quantized using oQ mixed-precision quantization.

Quantization details

• Model type: glm4_moe_lite
• Bits: 5
• Group size: 64
• Format: MLX safetensors

Benchmark

Model	File size	MMLU	JMMLU	HELLASWAG	GSM8K	ARC_CHALLENGE
GLM-4.7-Flash-REAP-23B-A3B-6bit	17.43 GB	62.3%	46.0%	53.0%	96.7%	73.0%
GLM-4.7-Flash-REAP-23B-A3B-oQ3	9.91 GB	53.3%	38.3%	47.7%	73.3%	73.3%
GLM-4.7-Flash-REAP-23B-A3B-oQ3.5	10.62 GB	57.7%	49.3%	60.0%	93.3%	65.0%
GLM-4.7-Flash-REAP-23B-A3B-oQ4	12.51 GB	59.3%	43.0%	53.3%	87.7%	78.7%
GLM-4.7-Flash-REAP-23B-A3B-oQ5	15.21 GB	61.0%	45.3%	59.0%	90.0%	81.0%

Detail

Model	Benchmark	Accuracy	Correct	Total	Time(s)
GLM-4.7-Flash-REAP-23B-A3B-6bit	MMLU	62.3%	187	300	505.9
GLM-4.7-Flash-REAP-23B-A3B-6bit	JMMLU	46.0%	138	300	239.7
GLM-4.7-Flash-REAP-23B-A3B-6bit	HELLASWAG	53.0%	53	100	114.7
GLM-4.7-Flash-REAP-23B-A3B-6bit	GSM8K	96.7%	29	30	88.6
GLM-4.7-Flash-REAP-23B-A3B-6bit	ARC_CHALLENGE	73.0%	73	100	64.4
GLM-4.7-Flash-REAP-23B-A3B-oQ3	MMLU	53.3%	160	300	602.7
GLM-4.7-Flash-REAP-23B-A3B-oQ3	JMMLU	38.3%	115	300	255.7
GLM-4.7-Flash-REAP-23B-A3B-oQ3	HELLASWAG	47.7%	143	300	346.8
GLM-4.7-Flash-REAP-23B-A3B-oQ3	ARC_CHALLENGE	73.3%	220	300	204.8
GLM-4.7-Flash-REAP-23B-A3B-oQ3	GSM8K	73.3%	220	300	1029.3
GLM-4.7-Flash-REAP-23B-A3B-oQ3.5	MMLU	57.7%	173	300	555.1
GLM-4.7-Flash-REAP-23B-A3B-oQ3.5	JMMLU	49.3%	148	300	252.4
GLM-4.7-Flash-REAP-23B-A3B-oQ3.5	HELLASWAG	60.0%	60	100	107.2
GLM-4.7-Flash-REAP-23B-A3B-oQ3.5	GSM8K	93.3%	28	30	76.4
GLM-4.7-Flash-REAP-23B-A3B-oQ3.5	ARC_CHALLENGE	65.0%	65	100	61.5
GLM-4.7-Flash-REAP-23B-A3B-oQ4	MMLU	63.3%	190	300	550.7
GLM-4.7-Flash-REAP-23B-A3B-oQ4	JMMLU	39.7%	119	300	250.9
GLM-4.7-Flash-REAP-23B-A3B-oQ4	HELLASWAG	51.0%	51	100	103.4
GLM-4.7-Flash-REAP-23B-A3B-oQ4	GSM8K	90.0%	27	30	75.6
GLM-4.7-Flash-REAP-23B-A3B-oQ4	ARC_CHALLENGE	70.0%	70	100	59.8
GLM-4.7-Flash-REAP-23B-A3B-oQ4	MMLU	59.3%	178	300	547.7
GLM-4.7-Flash-REAP-23B-A3B-oQ4	JMMLU	43.0%	129	300	232.6
GLM-4.7-Flash-REAP-23B-A3B-oQ4	HELLASWAG	53.3%	160	300	300.5
GLM-4.7-Flash-REAP-23B-A3B-oQ4	ARC_CHALLENGE	78.7%	236	300	179.7
GLM-4.7-Flash-REAP-23B-A3B-oQ4	GSM8K	87.7%	263	300	748.4
GLM-4.7-Flash-REAP-23B-A3B-oQ5	MMLU	61.0%	183	300	617.8
GLM-4.7-Flash-REAP-23B-A3B-oQ5	JMMLU	45.3%	136	300	273
GLM-4.7-Flash-REAP-23B-A3B-oQ5	HELLASWAG	59.0%	177	300	353.6
GLM-4.7-Flash-REAP-23B-A3B-oQ5	ARC_CHALLENGE	81.0%	243	300	201.2
GLM-4.7-Flash-REAP-23B-A3B-oQ5	GSM8K	90.0%	270	300	1001.1