README.md

metadata

license: apache-2.0
base_model: Guilherme34/Firefly-v4
library_name: gguf
language:
  - en
tags:
  - quantllm
  - gguf
  - llama-cpp
  - quantized
  - transformers
  - q4_k_m

🦙 Firefly-v4-gguf-q4

Guilherme34/Firefly-v4 converted to GGUF format

⭐ Star QuantLLM on GitHub

---

📖 About This Model

This model is Guilherme34/Firefly-v4 converted to GGUF format for use with llama.cpp, Ollama, LM Studio, and other compatible inference engines.

Property	Value
Base Model	Guilherme34/Firefly-v4
Format	GGUF
Quantization	Q4_K_M
License	apache-2.0
Created With	QuantLLM

🚀 Quick Start

Option 1: Python (llama-cpp-python)

from llama_cpp import Llama

# Load the model
llm = Llama.from_pretrained(
    repo_id="Guilherme34/Firefly-v4-gguf-q4",
    filename="Firefly-v4-gguf-q4.Q4_K_M.gguf",
)

# Generate text
output = llm(
    "Write a short story about a robot learning to paint:",
    max_tokens=256,
    echo=True
)
print(output["choices"][0]["text"])

Option 2: Ollama

# Download the model
huggingface-cli download Guilherme34/Firefly-v4-gguf-q4 Firefly-v4-gguf-q4.Q4_K_M.gguf --local-dir .

# Create Modelfile
echo 'FROM ./Firefly-v4-gguf-q4.Q4_K_M.gguf' > Modelfile

# Import to Ollama
ollama create Firefly-v4-gguf-q4 -f Modelfile

# Chat with the model
ollama run Firefly-v4-gguf-q4

Option 3: LM Studio

1. Download the .gguf file from the Files tab above
2. Open LM Studio → My Models → Add Model
3. Select the downloaded file
4. Start chatting!

Option 4: llama.cpp CLI

# Download
huggingface-cli download Guilherme34/Firefly-v4-gguf-q4 Firefly-v4-gguf-q4.Q4_K_M.gguf --local-dir .

# Run inference
./llama-cli -m Firefly-v4-gguf-q4.Q4_K_M.gguf -p "Hello! " -n 128

📊 Model Details

Property	Value
Original Model	Guilherme34/Firefly-v4
Format	GGUF
Quantization	Q4_K_M
License	apache-2.0
Export Date	2026-04-08
Exported By	QuantLLM v2.0

📦 Quantization Details

This model uses Q4_K_M quantization:

Property	Value
Type	Q4_K_M
Bits	4-bit
Quality	🟢 ⭐ Recommended - Best quality/size balance

All Available GGUF Quantizations

Type	Bits	Quality	Best For
Q2_K	2-bit	🔴 Lowest	Extreme size constraints
Q3_K_M	3-bit	🟠 Low	Very limited memory
Q4_K_M	4-bit	🟢 Good	Most users ⭐
Q5_K_M	5-bit	🟢 High	Quality-focused
Q6_K	6-bit	🔵 Very High	Near-original
Q8_0	8-bit	🔵 Excellent	Maximum quality

---

🚀 Created with QuantLLM

Convert any model to GGUF, ONNX, or MLX in one line!

from quantllm import turbo

# Load any HuggingFace model
model = turbo("Guilherme34/Firefly-v4")

# Export to any format
model.export("gguf", quantization="Q4_K_M")

# Push to HuggingFace
model.push("your-repo", format="gguf")

📚 Documentation · 🐛 Report Issue · 💡 Request Feature