gotta start torrenting models
mpasila
joined 1 year ago
Did anyone manage to get them working? I tried GGUF/GPTQ and running then unquantized with trust-remote-code and they just produced garbage. (I did try removing BOS tokens and still same thing)
They did already show a 70B model at some event few days ago https://techcrunch.com/2023/11/09/theres-something-going-on-with-ai-startups-in-france/ so maybe that's the premium one??
the devs mentioned that the 600B model takes about 1,3TB space alone..