LocalLLaMA

4 readers

4 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 2 years ago

MODERATORS

communick@poweruser.forum

Is there a technical reason that distributed LLMs don't exist? (alien.top)

submitted 2 years ago by chinawcswing@alien.top to c/localllama@poweruser.forum

31 comments fedilink hide all child comments

Why is there no analog to napster/bittorent/bitcoin with LLMs?

Is there a technical reason that there is not some kind of open source LLM that we can all install on our local host which contributes computing power to answering prompts, and rewards those who contribute computing power by allowing them to enter more prompts?

Obviously, there must be a technical reason which prevents distributed LLMs or else it would have already been created by now.

you are viewing a single comment's thread
view the rest of the comments

[–] dobkeratops@alien.top 1 points 2 years ago (3 children)

I've seen projects along these lines getting going, it's coming.

EDIT I hah didn't know Petals existed either.

there's going to be multiple attempts

[–] deviantkindle@alien.top 1 points 2 years ago (2 children)

I was thinking of distributed MoEs as well.

Question I have is how do you route queries? I don't know how to do that if all the Es are in the same cluster let alone distrivuted.

[–] madmax_br5@alien.top 1 points 2 years ago

I don’t know if there’s much value there when LORA’s are easily portable — you can just select the right lora as needed. One base model instance on one machine, many potential experts. This has been demonstrated.

load more comments (1 replies)