nirca/nirca-mini-GGUF

These GGUFs are an approximation of Nirca Mini, not the model itself. They run in stock llama.cpp, and the price is a fixed number of passes and an adapter that imitates part of the model.

1. Fixed 8 passes per byte. Nirca Mini runs one block again and again, 1 to 24 passes per byte, and a halting head decides per byte when to stop (a mean of 8 in training; on our parity prompts it stopped at pass 8 or 9 for 81 percent of bytes (base) and 85 percent (chat)). A GGUF has no loop, so these files hold exactly 8 unrolled passes and no halting. Every byte gets 8 passes, whatever the model would have chosen.
2. The architecture name is a stand-in. The files declare gpt-oss (llama.cpp's openai-moe) so that unmodified llama.cpp loads them. This is not an OpenAI model. Besides the model's 2 dense layers and 8 copies of its recurrent block, the file holds 8 extra "adapter" layers (18 layers in all, a stream 1,056 wide) that rebuild, from stock parts, how the model mixes its state between passes. That rebuilding is itself approximate.
3. How close it is (checked 2026-11-12, 8 prompts, teacher-forced next-byte, LM Studio's llama.cpp runtime 2.72.0 on Apple Metal, against the real model's own predictions): the next byte agrees on 92.4 percent of positions (base) and 89.8 percent (chat); KL divergence from the real model is 0.0415 (base) and 0.0870 (chat) nats; greedy replies are identical to the real model's on 1 of 8 prompts (base and chat). Replies are coherent, and they are not the replies the real model gives.
4. Size. About 3.0 GB per file for a model of about 109 million parameters, because the shared block is stored once per pass. The model's experts are q4_0, the adapter layers' experts q8_0, and attention, embeddings and the output layer f16 (norms and biases f32).
5. For the real model (halting, 1 to 24 passes), use the weights in nirca/nirca-mini or its WebGPU demo.

Not tested: LM Studio the app, any runtime other than llama.cpp, Linux, Windows, CUDA, Vulkan. The chat file was run on runtime 2.72.0 Metal only; the base file on runtimes 2.43.0 to 2.72.0, and on CPU.

Nirca Mini Details

Nirca Mini is a small chat model trained from scratch, with ternary weights (-1, 0 or +1, with a scale and a bias for each block of 128 weights). It reads and writes bytes, so it needs no tokenizer; these files carry a byte vocabulary of 265 entries (the 256 byte values and 9 special tokens). The full card is nirca/nirca-mini, which also holds the training-data and workflow sections for each line, written by program from each checkpoint's record.

Files

Both are built, with no retraining, from checkpoints published in nirca/nirca-mini. They do not update themselves when a new checkpoint is published there.

File	Line	Step	Tokens seen	Source model.safetensors sha256	GGUF sha256
nirca-mini-chat-stock-8pass-q4_0.gguf	Nirca Mini 0 (chat)	124,366	4,641,084,509	5291891335706844530f8120cb0a44c1d4baf45678b780593dad0ceb7c172d1e	53bc91254f7baf196d408af139fdb24e3acbb8a8ee5e0407132939e779b44fb6
nirca-mini-base-stock-8pass-q4_0.gguf	Nirca Mini 0 (base)	123,009	4,612,237,558	162b4e18d8d29b25866deb3a87d35576243a09db80ad74d1531da1c6cde7e0ce	a86c022c8f10b01bd5051b0ad3636533b2f3ec7a755e3e9bf0a8b3af4b35006a

How to use

Run either file in stock llama.cpp (this was run with LM Studio's llama-server 2.72.0 on Apple silicon, 2026-11-12, on both files):

llama-server -m nirca-mini-chat-stock-8pass-q4_0.gguf -c 2048 -ngl 99

The chat file carries the ChatML template. The model writes <|im_end|> as plain bytes, so llama.cpp does not know it is a stop token: send "stop": ["<|im_end|>"] with each chat request.

curl http://127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Hello"}],"temperature":0,"stop":["<|im_end|>"]}'

The base file is for raw text, with no template: send a prompt to /completion. The context is 2,048 bytes.

Model details

• Type: decoder-only language model, chat-tuned (chat file), with latent recurrence and a mixture of experts
• Parameters: about 109 million in the real model; the GGUFs repeat the shared block 8 times
• Vocabulary: 265 tokens, bytes
• Context length: 2,048 bytes
• Width: 512 (1,056 in the GGUF's stream), 8 attention heads, rotary position embeddings
• Experts: 32 in one shared pool, 8 active per token
• Chat format: ChatML (<|im_start|> / <|im_end|>)
• Language: English

Its training data includes text under CC BY 4.0 and CC BY-NC-SA 4.0.

Uses

• Intended: research and experimentation with small, ternary, recurrent models; running a small chat model in llama.cpp.
• Out of scope: anything where a wrong answer matters. It is not a source of facts or advice, and it has not been through any safety tuning.

Bias, risks, and limitations

It is a small research model, and it is often wrong or incoherent; the GGUFs add the approximation above on top of that. It may repeat biases in its training data. It is trained mostly on English, and it has no knowledge of events beyond what its data held. Verify anything it says.

Citation

Nirca Mini's architecture is derived from mini-AGI:

@misc{volotat_miniagi,
  author       = {volotat},
  title        = {mini-AGI},
  howpublished = {\url{https://github.com/volotat/mini-AGI}},
  note         = {GitHub repository}
}