this post was submitted on 18 Nov 2023
1 points (100.0% liked)

LocalLLaMA

3 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago
MODERATORS
communick@poweruser.forum
1
Is it worth using a bunch of old GTX 10 series cards ( like 1060 1070 1080 ) for running local LLM? (alien.top)
submitted 11 months ago by dpak90@alien.top to c/localllama@poweruser.forum
10 comments fedilink hide all child comments
 

Was wondering if theres anyway to use a bunch of old equipment like this to build an at home crunch center for running your own LLM at home, and whether it would be worth it.

you are viewing a single comment's thread
view the rest of the comments
[–] BlissfulEternalLotus@alien.top 1 points 11 months ago

I wish they come up with some extendable tensor chips that can work with old laptops.

Currently only 7b is the only model we can run comfortably. Even for 13 b, it's slower and it needs quite a bit of adjustment.