this post was submitted on 20 Nov 2023
1 points (100.0% liked)
LocalLLaMA
1 readers
1 users here now
Community to discuss about Llama, the family of large language models created by Meta AI.
founded 10 months ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
My personal take is what are the use cases for user friendly local LLMs on mobile compared to higher performance llm closed models?
Privacy is the only serious benefit I can think of.
Latency is one thing with the internet.
Any model that can run locally doesn’t need a round trip to a datacenter. This can of course depending on computer power
At current capabilities it's faster to query server on the opposite hemisphere than to generate locally.
round trip latency of an http request (or grpc or whatever pick your poison) is utterly insignificant compared to the time it takes to run the inference process, even for the smallest models with the fastest inference