Tiny-LLM 25M NQ SFT
This repository contains a 25M-parameter decoder-only language model fine-tuned with supervised fine-tuning (SFT) on the sentence-transformers/natural-questions dataset.
Model Summary
- Base model: custom Tiny-LLM 25M decoder-only model
- Export format: Hugging Face Transformers-compatible causal language model
- Fine-tuning method: Unsloth LoRA / QLoRA, merged into a standalone model
- Training dataset:
sentence-transformers/natural-questions - Task: lightweight question-answering style text generation
Prompt Format
User: {question}
Assistant:
The model was fine-tuned to continue from that prompt into an answer.
Training Details
Dataset
- Source:
sentence-transformers/natural-questions - Total usable records: 100,231
- Training examples: 98,227
- Validation examples: 2,004
- Split seed: 1337
- Answer handling: verbatim answers with light normalization
Fine-Tuning Setup
- Framework: Unsloth
- Max sequence length: 512
- Epochs: 2
- Effective batch size: 32
- Final artifact: merged Hugging Face model
Intended Use
This model is intended for:
- lightweight QA experiments
- Tiny-LLM fine-tuning experiments
- small-scale text generation testing
- research and prototyping
Limitations
- This is a very small 25M model and may produce inaccurate, incomplete, or hallucinated answers.
- The model was fine-tuned on QA-style prompt-completion data and may not generalize well outside that format.
- The model is not suitable for high-stakes, safety-critical, legal, financial, or medical use.
Example Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "YOUR_USERNAME/tiny-llm-25m-nq-sft"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
prompt = "User: what is gravity?\nAssistant:"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
**inputs,
max_new_tokens=120,
temperature=0.2,
top_p=0.9,
do_sample=True,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
- Downloads last month
- 14