Excellent work.
Curious if tech could be used on Llama 3.1 8Bs?
and/or "older" Mistral 7Bs? Solar 10.7B?
Could push the older Llamas just that much further...;
Llama 3.2 3Bs ?
Again ; excellent work.
PS: Just built some 46B Mistrals 2506-Instructs ; gonna push the creativity to the max.
Yep, the method can be used to unslop any model that works with vllm.
From a quick 20 messages generation test at (XTC: 0, 0), I think the antislop might need some work;
https://gist.github.com/MerijnHendriks/63d95f7a608d83b896cd90790debee05
- stone: 21x
- rhythm: 17x
- intricate: 14x
- gentle: 13x
- chant: 13x
- adorn: 11x
- intricate patterns: 8x
- ornate: 7x
- tapping: 6x
Slop phrases:
- The motion smooth and practiced
- The day's possibilities stretch before you
- a stark contrast to
These jump out to me when reading the generated text.
In other generated messages with this model, popular pics are:
- delicate
- can't help but
- a testament to
- shiver(s) down (your/his/her/their) spine(s)
- unshed tears
From a quick 20 messages generation test at (XTC: 0, 0), I think the antislop might need some work;
https://gist.github.com/MerijnHendriks/63d95f7a608d83b896cd90790debee05
Oh, that looks a bit broken actually, with all that repetition. I might have to redo the finetune for this model.
@nohurry this version should be less overcooked: https://huggingface.co/sam-paech/Mistral-Small-3_2-24B-Instruct-2506-antislop.v2