"observer": "mse" could improve accuracy?
#1
by
CHNtentes
- opened
Yes, that is correct! However, mse observer would cost more gpu runtime to quantize, and considering the model large size, it would be comparably more expensive.
Yes, that is correct! However, mse observer would cost more gpu runtime to quantize, and considering the model large size, it would be comparably more expensive.
Thanks for your reply. I did not see much drop in quantize speed, but it was just a 4B model.
CHNtentes
changed discussion status to
closed