If you want it.
#4
by MrReclusive - opened
Hey, I have a fp4 mixed distilled on civitai.
https://civitai.com/models/2445970?modelVersionId=2751189
If you want to grab it, its the dist fp4 extreme. when i made it only a few things remained at bf16, most was quantized down to fp4, with the few things that fail at fp4 set to fp8 (first and last block also at fp8), it sits at 13gb on gpu while idle.