zai-org/GLM-4.6
Text Generation
•
357B
•
Updated
•
331k
•
•
1.13k
Carl Johnson
fluffyone
AI & ML interests
None yet
Recent Activity
liked
a model
about 1 month ago
zai-org/GLM-4.6
liked
a model
4 months ago
deepseek-ai/DeepSeek-V3.1
liked
a model
5 months ago
Qwen/Qwen3-235B-A22B-Instruct-2507
Organizations
None yet
liked
a
model
about 1 month ago
liked
a
model
4 months ago
liked
a
model
5 months ago
liked
2
models
7 months ago
upvoted
a
collection
10 months ago
liked
2
models
about 1 year ago
liked
a
Space
about 1 year ago
liked
2
models
about 1 year ago
liked
3
models
over 1 year ago
upvoted
a
collection
over 1 year ago
reacted to
Lewdiculous's
post with ❤️🚀
over 1 year ago
Post
62754
More context for your Pascal GPU or older!
Update: Now available in the official releases of KoboldCpp!
[releases] https://github.com/LostRuins/koboldcpp/releases/latest
These are great news for all the users with GTX 10XX, P40...
Flash Attention implementation for older NVIDIA GPUs without requiring Tensor Cores has come to llama.cpp in the last few days, and should be merged in the next version of KoboldCpp, you can already try it with another fork or by building it.
[Mentioned KCPP fork] https://github.com/Nexesenex/kobold.cpp/releases/latest
[PR] https://github.com/ggerganov/llama.cpp/pull/7188
You should expect less VRAM usage for the same context, allowing you to experience higher contexts with your current GPU.
There have also been reported final tokens/second speed improvements for inference, so that's also grand!
If you have tried it, I'd like to hear your experiences with --flashattention so far, especially for this implementation and for the large number of Pascal (GTX 10XX, P40...) cards.
Discussion linked bellow, with more links to relevant information:
https://huggingface.co/LWDCLS/LLM-Discussions/discussions/11
Cheers!
Update: Now available in the official releases of KoboldCpp!
[releases] https://github.com/LostRuins/koboldcpp/releases/latest
These are great news for all the users with GTX 10XX, P40...
Flash Attention implementation for older NVIDIA GPUs without requiring Tensor Cores has come to llama.cpp in the last few days, and should be merged in the next version of KoboldCpp, you can already try it with another fork or by building it.
[Mentioned KCPP fork] https://github.com/Nexesenex/kobold.cpp/releases/latest
[PR] https://github.com/ggerganov/llama.cpp/pull/7188
You should expect less VRAM usage for the same context, allowing you to experience higher contexts with your current GPU.
There have also been reported final tokens/second speed improvements for inference, so that's also grand!
If you have tried it, I'd like to hear your experiences with --flashattention so far, especially for this implementation and for the large number of Pascal (GTX 10XX, P40...) cards.
Discussion linked bellow, with more links to relevant information:
https://huggingface.co/LWDCLS/LLM-Discussions/discussions/11
Cheers!
- 24 replies
reacted to
Undi95's
post with 🔥
over 1 year ago
Post
16608
Hey everyone,
Just wanted to shout out a massive thank you to all 2000 of you who've followed me on Hugging Face! 🎉 It's incredible to have such an awesome crew backing me up as I dive into all these LLM experiments.
Even though not all my models turn out perfect, I've found some real gems and methods along the way 💎. It's like digging for treasure – sometimes you found nothing, but sometimes you find a pearl, and sometimes you find a new method to try.
Your support and encouragement mean the world to me, and I'm really stoked to keep experimenting and learning. If you told me some years ago I would have so much people following me for what I do, I wouldn't have believed it. Here's to more discoveries and adventures ahead! 🚀
Also, big thanks once again, and a huge shoutout to @IkariDev for being there through this journey and supporting me. I'm excited for our future work together and hope we will continue to make people happy! 👏
I want to thank @Gryphe too, since my early work was heavily inspired from MythoMax and the RP/ERP vibe of it. If I'm here today it's probably because of you 😂
I was so close to forget @chargoddard and his amazing tool too! What will we do without mergekit in our life? Thank you! 🙏
See y'all at 3k!
Just wanted to shout out a massive thank you to all 2000 of you who've followed me on Hugging Face! 🎉 It's incredible to have such an awesome crew backing me up as I dive into all these LLM experiments.
Even though not all my models turn out perfect, I've found some real gems and methods along the way 💎. It's like digging for treasure – sometimes you found nothing, but sometimes you find a pearl, and sometimes you find a new method to try.
Your support and encouragement mean the world to me, and I'm really stoked to keep experimenting and learning. If you told me some years ago I would have so much people following me for what I do, I wouldn't have believed it. Here's to more discoveries and adventures ahead! 🚀
Also, big thanks once again, and a huge shoutout to @IkariDev for being there through this journey and supporting me. I'm excited for our future work together and hope we will continue to make people happy! 👏
I want to thank @Gryphe too, since my early work was heavily inspired from MythoMax and the RP/ERP vibe of it. If I'm here today it's probably because of you 😂
I was so close to forget @chargoddard and his amazing tool too! What will we do without mergekit in our life? Thank you! 🙏
See y'all at 3k!
- 5 replies
liked
a
model
over 1 year ago