Llama-3.3-8B-Instruct-OmniWriter

An experiment in creating a creative, uncensored storyteller based on the "smart" Llama 3.3 8B Instruct base.

Training Specs

• Training Method: LoRA (16-bit)
• LoRA Rank (r): 32
• LoRA Alpha: 16
• Scaling Type: Rank-Stabilized LoRA (RS-LoRA)
• Batch Size: 32
• Gradient Accumulation: 2
• Epochs: 1
• Learning Rate: 2e-5
• Optimizer: AdamW (Fused)
• LR Scheduler: Cosine
• Noise Level: NEFTune (alpha=5)

Special Thanks

This fine-tune wouldn't be possible without the incredible work of the community:

• p-e-w for developing Heretic - an essential tool for censorship removal.
• allura-forge and shb777 for providing access to the Llama 3.3 8B weights.
• Gryphe for the meticulously curated Writing Prompts datasets (Opus and ChatGPT-4o versions).
• unsloth for their training acceleration framework that makes local fine-tuning fast and memory-efficient.
• AMD for their Instinct™ MI300X GPU.