Does anyone have any plans to quantizie any of the newly relased Gtp-sw3 models?
Or if there is a guide (very detailed) how to do it. Iam not very tech savy so what i have found/googled so far, is above my level. Which is quit low to be honest...
Is there anyone planing to quantizie these models or willing to do it?
The modelI Iam most interested in is the 20B instruct model. (Guff or AWQ).
https://huggingface.co/AI-Sweden-Models/gpt-sw3-20b-instruct
Replika.ai. its an app or if you like you can use web interface.