ubergarm ik_llama.cpp quant
#5
by
Fernanda24
- opened
after the best quants for kimi k2 you have me wondering will we see some ubergarm quants for this model :)
not sure if chat template jinja is fully optimized/fixed and what the best params are though
Thanks for letting me know about this GLM-4.5-Air finetune! Looks interesting, but I might not get to it. If you want some more info check out the Beaver AI Discord where they are discussing how to modify it (mtp layers) to convert it to GGUF and you can try out a mainline quant by bartowski to see how it is working for your specific use cases:
- https://huggingface.co/BeaverAI
- https://discord.com/channels/1238219753324281886/1443458411332370583/1443481185488736298
- https://huggingface.co/bartowski/PrimeIntellect_INTELLECT-3-GGUF
oh wait, you already posted over there lmao
https://huggingface.co/bartowski/PrimeIntellect_INTELLECT-3-GGUF/discussions/1