LocalLLaMA

3 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago

MODERATORS

communick@poweruser.forum

Is there a technical reason that distributed LLMs don't exist? (alien.top)

submitted 1 year ago by chinawcswing@alien.top to c/localllama@poweruser.forum

31 comments fedilink hide all child comments

Why is there no analog to napster/bittorent/bitcoin with LLMs?

Is there a technical reason that there is not some kind of open source LLM that we can all install on our local host which contributes computing power to answering prompts, and rewards those who contribute computing power by allowing them to enter more prompts?

Obviously, there must be a technical reason which prevents distributed LLMs or else it would have already been created by now.

you are viewing a single comment's thread
view the rest of the comments

[–] exoticCuriosity@alien.top 1 points 1 year ago (1 children)

I think the primary concern revolves around the security and data privacy of users, given the absence of assurance regarding the safety of your data when operating on a network accessible to anyone, including potential malicious actors.

I believe implementing an experimental model, wherein users can contribute their GPU for a specific model and receive credits in return, which they can use to use other models instead of downloading Terabytes worth of data

[–] sdmat@alien.top 1 points 1 year ago

No, the primary concern is that network latency kills the serial performance of LLMs.

You can have a distributed llm getting decent throughput in total across many slow generations. You can't have a distributed LLM with throughput for a single generation competitive to running in a single cluster.