metadata
base_model: unsloth/gemma-3-270m-it
library_name: peft
pipeline_tag: text-generation
license: gpl-3.0
tags:
- lora
- gemma
- resonate
- reasoning
- multilingual
language:
- en
- fr
- de
- ru
- he
- ar
- ja
- zh
Gemma-3 270M-IT /resonate/ LoRA
LoRA adapter that teaches Gemma-3 270M-IT the /resonate/ reasoning format — stream-of-consciousness thinking followed by a clean answer.
What is /resonate/?
/resonate/
[free-form thinking — cynical, multilingual, associative, honest]
/resonated/
[clean, structured answer]
The model learns to THINK before answering. The /resonate/ block is raw reasoning — it can switch languages, use metaphors, be irreverent. The /resonated/ block is the distilled answer.
Architecture
| Base | unsloth/gemma-3-270m-it (268.1M params) |
| Frozen | embed_tokens = 167.8M (63%) — all 140 languages preserved |
| LoRA | R=16, alpha=32, q_proj + v_proj only |
| Trainable | 0.74M (0.3% of total) |
| Training | 3 epochs, 6445 examples, 32 min on A100 |
| Best val loss | 2.9241 |
Key insight
Freezing embed_tokens (63% of the model) preserves the multilingual embedding space. The LoRA adapter only modifies attention projections — teaching the model HOW to think, not WHAT languages to know.
Languages verified working
English, French, German, Russian, Hebrew, Arabic, Japanese, Chinese — all generate coherent text with /resonate/ format after fine-tuning.
Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch
tokenizer = AutoTokenizer.from_pretrained("ataeff/g")
base = AutoModelForCausalLM.from_pretrained("unsloth/gemma-3-270m-it", dtype=torch.bfloat16)
model = PeftModel.from_pretrained(base, "ataeff/g")
prompt = "<start_of_turn>user\nWhat is the meaning of life?<end_of_turn>\n<start_of_turn>model\n"
ids = tokenizer(prompt, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=200, temperature=0.7, do_sample=True)
print(tokenizer.decode(out[0], skip_special_tokens=True))
Training data
resonance_yent_full.jsonl— 6435 examples of /resonate/ format dialoguesresonance_gold_10.jsonl— 10 hand-crafted gold examples (math, philosophy, code, multilingual)