LocalLLaMA

3 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago

MODERATORS

communick@poweruser.forum

Dual 3090,24GB & 1070 worth it? (alien.top)

submitted 11 months ago by BlueMetaMind@alien.top to c/localllama@poweruser.forum

5 comments fedilink hide all child comments

I upgraded my system a year ago. Amongst these was a GPU 1070 -> 3090 upgrade. My old card rusts now in the cellar. I heard people around here have mentioned they run desktop on their old card to completely free VRAM in their workhorse card.

is this worth doing in the context of loading LLMs and playing around with them, no training yet? The downside is a worse thermal situation if I cram them together.

top 5 comments

sorted by: hot top controversial new old

[–] nazihater3000@alien.top 1 points 11 months ago

You can use the 1070 to run the desktop, stable diffusion, Whisper, TTS and other add-ons.

[–] panchovix@alien.top 1 points 11 months ago

it will work but you will be limited to 1070 speeds if using all 3 gpus

[–] 0xd00d@alien.top 1 points 11 months ago

Be sure to prioritize the 3090s pcie lanes

[–] A_for_Anonymous@alien.top 1 points 11 months ago (1 children)

I just use Linux. Stop the X server or don't install it altogether. (If you have it, just Ctrl+Alt+F1, log in, sudo systemctl stop lightdm.) Enjoy the entirety of your VRAM minus 1 MB available for you, so from there you run oobabooga or an API server or whatever and connect to it from your laptop.

As an added benefit, I can leave the GPU and all the noisy fans and their heat somewhere else in my home. And you don't need to connect a display to the Linux box either, just setup openssh-server, keys and work remotely.

[–] BlueMetaMind@alien.top 1 points 11 months ago

Clever. I often run SD and textUI in --listen mode than use it on my IPad.