ModeradorDoFariaLima

joined 10 months ago
sorted by: new top controversial old
How to upgrade to the next VRAM breakpoints, and is it worth it? in c/localllama@poweruser.forum
[–] ModeradorDoFariaLima@alien.top 1 points 9 months ago

Wish we could just solder more VRAM to the cards. Such a silly thing to keep holding us back.

OpenAi was developing a very powerful model that could threat human existence, was reason behind Altmans removal. in c/localllama@poweruser.forum
[–] ModeradorDoFariaLima@alien.top 1 points 9 months ago

Open ai does marketing better than they make their models.

Are 7b models useful? in c/localllama@poweruser.forum
[–] ModeradorDoFariaLima@alien.top 1 points 10 months ago

I use airoboros 7b to give me plots and write the beginnings of stories. It's very useful for that.

I’m extremely confused about system requirements. Some people are worried about ram and others about vram. I have 64gb of ram and 12gb vram. What size of model can I run? in c/localllama@poweruser.forum
[–] ModeradorDoFariaLima@alien.top 1 points 10 months ago

Imo running any AI on anything other than VRAM makes it so slow it's unusable. So, you can play around with 13b quantized models.

ExLlamaV2: The Fastest Library to Run LLMs in c/localllama@poweruser.forum
[–] ModeradorDoFariaLima@alien.top 1 points 10 months ago (1 children)

Too bad I think that Windows support for it was lacking (at least, last time I checked it). It needs a separate thing to make it work properly, and this thing was only for Linux.