LocalLLaMA

1 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago

MODERATORS

communick@poweruser.forum

Should I be able to run lzlv_Q4_K_M.gguf with 128gb cpu ram? It keeps erroring out. (alien.top)

submitted 10 months ago by Brad12d3@alien.top to c/localllama@poweruser.forum

2 comments fedilink hide all child comments

I'm still new to this and I thought that 128gb CPU ram would be enough to run a 70b model? I also have an RTX 4090. However, everytime I try to run lzlv_Q4_K_M.gguf in Text Generation UI, I get "connection errored out". Could there be a setting that I should tinker with?

you are viewing a single comment's thread
view the rest of the comments

[–] Herr_Drosselmeyer@alien.top 1 points 10 months ago

It should work with those specs. Not sure what "connection" it means. Perhaps post a screenshot of the console?

permalink
fedilink
source