Excited for incoming uploads!
#1
by
ubergarm - opened
Heya AesSedai, thanks for making such great custom mainline llama.cpp compatible quants!
Your work measuring not just perplexity, but going the extra mile with KLD statistics, is top notch research and helps inform and empower community users to choose the quants that best fit their specific hardware and workload needs.
Cheers!
Uploads complete!
Need to test these. Been loving the UD Q3 quant of minimax m2.5 but he hallucinates a bit too often (not terrible but weird things like a random chinese letter and grammar mistakes). Hope one of these that can fit on a 128gb strix halo can be much better.