More Quantization options

#3
by kkalyfo - opened

Please provide Q8 quantization too!

Athena Research Center | Institute for Language and Speech Processing org

Sure, we'll soon update the GGUF model with Q8 as well.

I have Q8 up here if you need it before they get to it: https://huggingface.co/bartowski/ilsp_Llama-Krikri-8B-Instruct-GGUF

Thanks! Is it with the updated weights?

should be yes, it was created after the most recent update to their relevant files

though that gives me an idea that i should probably include the latest commit hash that the quants are made from just for extra clarity πŸ€”

Thanks for your response!

Athena Research Center | Institute for Language and Speech Processing org

We have updated the model with the last quants (q8, q6 and q3).

Great, thanks!

Sign up or log in to comment