Quantized using Intel's AutoRound quantization tool.

auto-round-best --model facebook/opt-13b --scheme "w4a16"

Safetensors

Model size

0.3B params

Tensor type

I32

F16

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Emanresu/opt-13b-w4g128-AutoRound

Base model

Quantized

(2)

this model