Llama 3
Collection
SWAN quantized versions of Llama 3.1 and 3.3 70B Instruct (MLX) • 2 items • Updated
Llama-3.3-70B-Instruct — 46GB (MLX)
Mixed-precision quantized version of meta-llama/Llama-3.3-70B-Instruct optimised by baa.ai using a proprietary Black Sheep AI method.
Per-tensor bit-width allocation via advanced sensitivity analysis and budget-constrained optimisation — no calibration data required.
Metrics
| Metric | Value |
|---|---|
| Size | 46 GB |
| Average bits | 5.6 |
| WikiText-2 PPL (median) | 4.3544 |
| MMLU vs BF16 | 94.8% of BF16 |
Usage
from mlx_lm import load, generate
model, tokenizer = load("baa-ai/Llama-3.3-70B-Instruct-RAM-50GB-MLX")
response = generate(model, tokenizer, prompt="Hello!", max_tokens=256)
print(response)
Quantized by baa.ai
- Downloads last month
- 90
Model size
71B params
Tensor type
BF16
·
U32 ·
Hardware compatibility
Log In to add your hardware
4-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for baa-ai/Llama-3.3-70B-Instruct-RAM-50GB-MLX
Base model
meta-llama/Llama-3.1-70B Finetuned
meta-llama/Llama-3.3-70B-Instruct