LocalLLaMA

3 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago

MODERATORS

communick@poweruser.forum

Dual 3090,24GB & 1070 worth it? (alien.top)

submitted 11 months ago by BlueMetaMind@alien.top to c/localllama@poweruser.forum

5 comments fedilink hide all child comments

I upgraded my system a year ago. Amongst these was a GPU 1070 -> 3090 upgrade. My old card rusts now in the cellar. I heard people around here have mentioned they run desktop on their old card to completely free VRAM in their workhorse card.

is this worth doing in the context of loading LLMs and playing around with them, no training yet? The downside is a worse thermal situation if I cram them together.

you are viewing a single comment's thread
view the rest of the comments

[–] A_for_Anonymous@alien.top 1 points 11 months ago (1 children)

I just use Linux. Stop the X server or don't install it altogether. (If you have it, just Ctrl+Alt+F1, log in, sudo systemctl stop lightdm.) Enjoy the entirety of your VRAM minus 1 MB available for you, so from there you run oobabooga or an API server or whatever and connect to it from your laptop.

As an added benefit, I can leave the GPU and all the noisy fans and their heat somewhere else in my home. And you don't need to connect a display to the Linux box either, just setup openssh-server, keys and work remotely.

[–] BlueMetaMind@alien.top 1 points 11 months ago

Clever. I often run SD and textUI in --listen mode than use it on my IPad.