ASHQ1 — Autonomous Selective Hybrid Quantization v 1

👋 I'm back. Active development has resumed — expect improvements soon.

⚠️ Experimental. ASHQ1 is a personal research project that I will be refining over time. Use at your own risk. Results may vary between architectures and fine-tunes. Feedback and contributions welcome.

Latest update (v7): MTP detection now catches the whole head (not just nextn.*), output/embd are pinned at Q5_K outside the budget, --allow-q3-or-lower actually covers its documented types, sub-4-bit toxicity (TOXICITY_SUB4) fights the low-bit disease, MoE routers are pinned at F16, and there's a new --top-down mode that starts everything at F16 and downgrades. New architectures: granite (Granite-4.2) and bailingmoe3 (Ling-3.0 MoE), detected via general.architecture metadata. Tested on Ornith-1.5-9B: PPL 8.6341 @ 6511 MiB.

ASHQ1 is a post-training quantization method for GGUF models that uses an imatrix-driven priority queue to maximise theoretical quality per megabyte. Instead of uniform bit-depth or heuristic layer-blocking, it treats tied tensor groups as monolithic entities and greedily upgrades them by strict mathematical utility — the product of summed importance and theoretical MSE reduction, divided by size cost.

Results

Method	Model	Size	PPL (ctx 1024)	Δ vs Uniform
ASHQ1 (v7)	Ornith-1.5-9B-MTP	6511 MiB	8.6341 ± 0.06112	—
ASHQ1 top-down (v7)	Ornith-1.5-9B-MTP	6509 MiB	8.6337 ± 0.06111	—
Remix Quality-36pc	Ornith-1.5-9B (no MTP)	6330 MiB	9.3692	+0.7351
ASHQ1 (v7)	Ornith-1.0-9B-MTP	6011 MiB	7.4830 ± 0.04876	−0.1418
Uniform Q6_K	Ornith-1.0-9B-MTP	7198 MiB	7.6248 ± 0.05039	baseline

v7 holds the v6 line on 1.0 (7.4830 vs 7.4697 — inside noise) while fixing MTP handling and embd/output tiers. Historical v6 point: 6012 MiB / 7.4697.

On Ornith-1.5, one 6.5 GB ASHQ1 quant beats the entire ASHQ1-Remix ladder (best: 9.3692 @ 6330 MiB; even their 9.4 GB Fidelity tier at 9.5239). Note the Remix numbers are measured without MTP heads; ours keep the full MTP head at Q8_0 inside the file.

ASHQ1 is often on par with hand-tuned SHQ quants in quality, and sometimes surpasses them. At the same time, it saves significant time and effort — just set your target size and go.

Real-World Validation

Beyond PPL: Ornith-1.0 ASHQ1 6500 (6.4 GB) drove Pi, an autonomous coding agent, to build a complete ~1100-line finance-dashboard app from scratch — planning, codegen, self-debugging, validation — at temperature 0.6, first run, no cherry-picking.

How It Works

1. Floor Assignment

Every tensor starts at a minimum tier by class. SSM params and norms lock at F16. output/token_embd are pinned at Q5_K outside the budget (llama.cpp writes them at the output/token types unconditionally, so the estimator counts them there too). Weight matrices start at Q4_K (or IQ4_XS for QAT models, except attention). MTP heads deploy at Q8_0.

With --allow-q3-or-lower, CAN_Q3 types (ffn_gate, ffn_up, ffn_down, attn_output, ssm_out) start as low as IQ2_XXS, giving the priority queue more room to upgrade important tensors. Tensors missing imatrix data are kept at Q4_K to avoid garbage at low bitrates.

2. Importance

Imatrix in_sum2 measures how much each weight contributes to the output variance. Layer position weighting was tested but showed no PPL benefit and has been removed.

3. Tied Group Detection

Tensors with numerically identical in_sum2 arrays are tied (shared weights). They form a single upgrade group — all members upgrade together as one unit. Group importance is the sum of its members' importance, preventing large groups from being starved of budget.

4. Priority Queue Drain

All possible single-tier upgrades are pushed into a max-heap:

utility/MiB = sum(timp[group]) × (MSE(cur) − MSE(next)) / (size(next) − size(cur))

MSE per tier is theoretical: MSE = 2^(-2 × bpw). K-quants get +0.1 effective bpw vs IQ-quants at the same real bpw (IQ4_NL = 4.40), so IQ4_NL→Q4_K is a free quality gain. Every tier's effective bpw is strictly increasing — equal values would strand both greedy chains at zero-gain steps.

Sub-4-bit toxicity: tensors below 4.0 real bpw poison PPL superlinearly ("low-bit disease", empirically confirmed), so their effective MSE is inflated ×2 (TOXICITY_SUB4). The queue escapes sub-4 eagerly bottom-up and enters it reluctantly top-down. Measured on Ornith-1.5 @4500: 99 → 79 sub-4-bit tensors, PPL 10.1077 → 9.9629 at the same size.

The queue pops the highest-utility upgrade, applies it, pushes the next upgrade for that group, and drains until the budget is exhausted. Zero-cost upgrades carry infinite utility, so they apply as free upgrades inline during the normal drain.

5. Top-down mode (--top-down)

Reverse direction: literally every tensor except the structural pins (MTP → Q8_0, output/embd → Q5_K, MoE routers → F16) starts at F16 — norms, 1D tensors, biases, all of it — and is greedily downgraded, cheapest Σ(importance) × ΔMSE / saved_MiB first, until under target. Downgrades stop at the same base floors bottom-up starts from. On Ornith-1.5 @6500 every norm sank to the Q4_K floor and PPL didn't move (8.6337 vs 8.6341, σ=0.06): norms on this architecture are free real estate. Note the estimate assumes the binary quantizes 1D/*_norm.weight too, while llama.cpp physically keeps them at F16 — harmless here (they weigh ~2 MiB total) but be aware of it at other budgets.

Duel on Ling-3.0-tiny @5000 (same budget, opposite allocations):

Tier	Bottom-up (4861 MiB, PPL 13.0681)	Top-down (4999 MiB, PPL 13.1094)
F16	215 / 9.7 MiB	126 / 222.8 MiB
Q4_K	43 / 2271 MiB	210 / 2059 MiB
Q5_K	30 / 2036 MiB	36 / 2304 MiB
Q6_K	8 / 94 MiB	14 / 109 MiB
Q8_0	230 / 432 MiB	140 / 285 MiB

Bottom-up spreads upgrades wide; top-down keeps the most precious tensors (incl. top-importance shared experts) at F16 and crushes the rest to the floor. PPL difference: 0.04 at σ=0.10 — indistinguishable. Direction doesn't matter; final bit placement does. Full write-up in the Ling model card.

Phase 2 — slack filling (v7.1). Top-down used to stop at the first fit, burning the leftover (up to ~150 MiB on MoE). Now the same mirrored imp×ΔMSE/cost queue (ceiling F16) spends the slack on upgrades — the last-downgraded groups recover first. Slack is now exactly 0.0 on all tested configs. Measured on Ling-3.0-tiny @6000: PPL 12.9802 vs 13.0001 before (Δ=−0.02, σ=0.10) — no quality change, pure budget discipline.

Why It Works

Problem	ASHQ1 Solution
Uniform quant wastes bits on low-importance tensors	Priority queue allocates budget where it matters
Heuristic hand-tuning doesn't scale	Single knob: --size in MiB
Hand-tuned SHQ hybrids need days of PPL sweeps	Queue converges in ~1 sec for any budget
Large tied groups starved by per-tensor logic	sum(timp) prevents 32× group penalty
IQ4_NL→Q4_K at same bpw is a no-op	Infinite-utility free upgrades apply inline during the drain
No PPL-per-budget curve needed	Queue optimises for MSE directly
Tensors without imatrix crash at low bitrates	has_imatrix check falls back to Q4_K floor

Supported Architectures

Arch	Detection	Features
qwen35	SSM + QKV	Hybrid attention, SSM layers, GQA, MTP support
mellum2	MoE (exps tensors)	Mixture of Experts, GQA, routers pinned F16
bailingmoe3	KDA+MLA + MoE	Ling-3.0 family: hybrid-linear attention, 128 routed + 1 shared expert (shared experts mapped high, routers pinned F16)
granite	general.architecture	Dense GQA, 40 layers, separate Q/K/V (NOT qat)
gemma4	Layer-scale norms	QAT support, Q4_K attention floor

Detection prefers general.architecture metadata and falls back to tensor-name heuristics.

MTP (Multi-Token Prediction) heads are handled explicitly: the MTP layer is excluded from n_layers at read time, so the whole head (blk.32.* plus nextn.*) deploys at Q8_0 and is excluded from the classifier's budget (its cost is subtracted from the target upfront).

New architectures can be added via ARCH_FEATURES in constants.py.

Code Structure

File	Role
main.py	CLI entry point, orchestration, --show-floors, multiple --imatrix support
model_reader.py	Reads GGUF, detects architecture/prefix/n_layers/MTP at runtime
imatrix_reader.py	Parses imatrix GGUF, detects tied groups via np.allclose(in_sum2), combines multiple imatrix
classifier.py	Base floors → tied group building → priority queue drain (bottom-up) or greedy downgrade (top-down --top-down)
config_generator.py	Generates --tensor-type regex rules from classified tensors (valid ECMAScript regex with pipe-alternated ranges)
quantizer.py	Subprocess wrapper around llama-quantize (binary via LLAMA_QUANTIZE env)
constants.py	TENSOR_CLASS mapping, CLASS_MAX_TIER, CAN_Q3, MSE_BPW, TIER_BPW, ARCH_FEATURES
utils.py	Dry-run size parsing

Usage

Quantization

pip install -r requirements.txt

# Dry run (∼1 sec)
python main.py --model model.gguf --imatrix imatrix.gguf --size 6800

# Actual quant (∼10 min)
python main.py --model model.gguf --imatrix imatrix.gguf --size 6800 --run

# Show hard floors
python main.py --show-floors

# Multiple imatrix (combined with max/mean)
python main.py --model model.gguf --imatrix i1.gguf --imatrix i2.gguf \
  --imatrix-method max --size 6800 --run

# Allow low-bit tensors (IQ2_XXS through Q8_0 spread)
python main.py --model model.gguf --imatrix imatrix.gguf --size 6000 \
  --allow-q3-or-lower --run

# Top-down: everything from F16, downgrade to fit (norms included)
python main.py --model model.gguf --imatrix imatrix.gguf --size 6500 \
  --top-down --run

The llama-quantize binary defaults to the local build, overridable via the LLAMA_QUANTIZE env var. If even the base floors don't fit the target, the run aborts with an error instead of silently producing an oversized quant. With multiple --imatrix files the importance table uses the combination, but the binary itself only receives the first file (warned at runtime).

Inference (llama-server)

Recommended server flags for serving ASHQ1 quants:

./build/bin/llama-server \
  -m model-ASHQ1.gguf \
  -c 50000 \
  --jinja \
  -fit off \
  -ngl 99 \
  --flash-attn on \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  --port 8080 \
  --mmap \
  --temp 1.0 \
  --top-p 0.95 \
  --min-p 0 \
  --top-k 20 \
  --seed -1 \
  --parallel 1

Tier Reference

Tier	BPW	MSE_BPW
F16	16.0	16.0
Q8_0	8.50	8.50
Q6_K	6.5625	6.5625
Q5_K	5.50	5.50
Q4_K	4.50	4.50
IQ4_NL	4.50	4.40
IQ4_XS	4.25	4.25
Q3_K	3.4375	3.4375
IQ3_M	3.66	—
IQ3_S	3.44	3.44
IQ3_XXS	3.0625	3.0625
IQ2_S	2.50	2.50
IQ2_XS	2.3125	2.3125
IQ2_XXS	2.0625	2.0625
IQ1_S	1.5625	1.5625

All effective values are strictly increasing — this is load-bearing: equal values would strand both greedy chains at zero-gain steps (see MSE_BPW in classifier.py).

Quantization Configs

Generated configs are valid llama-quantize arguments with ECMAScript-compatible regex patterns. Each --tensor-type rule matches a group of tensors that share the same target tier, with layers grouped into contiguous ranges:

• (blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k=Q8_0 — specific attention layers at Q8_0
• (blk|BLK)\.((?:22|23|24|25|26))\.ffn_gate=Q6_K — range of FFN layers at Q6_K
• .*output_norm.*=F16 — global catch-all

Rules are sorted by specificity (specific layers, high tiers first) because llama-quantize uses first-match-wins.

Model repos

• Ornith-1.5/1.0 MTP quants + configs: docs/README-Ornith-MTP-1.5.md
• Older cards: docs/README-MTP-repo.md, docs/README-Ornith.md

References

• ASHQ1 repo / GitHub
• GGUF specification
• llama.cpp