Thanks was looking for base model quants.
#1
by RyanoSaurus-Wrex - opened
Did you do intermediate conversion from safetensors to gguf in native bf16 or clip it to old fp16? I was worried because you only uploaded fp16 for largest gguf and mm projector?
I went straight from safetensors to F16. I didn't think it would make much difference, but it seems I was wrong. I will have another go at it.
Edit: done!
Yeah, it makes big difference, clipping damage is so rampant in AI conversion. Most models are trained at BF16 now and people still do intermediate conversions to fp16 and massively damage the models right off the bat. Thanks so much!
RyanoSaurus-Wrex changed discussion status to closed