this post was submitted on 20 Nov 2023
1 points (100.0% liked)
LocalLLaMA
1 readers
1 users here now
Community to discuss about Llama, the family of large language models created by Meta AI.
founded 10 months ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
My personal take is what are the use cases for user friendly local LLMs on mobile compared to higher performance llm closed models?
Privacy is the only serious benefit I can think of.
Latency is one thing with the internet.
Any model that can run locally doesn’t need a round trip to a datacenter. This can of course depending on computer power
round trip latency of an http request (or grpc or whatever pick your poison) is utterly insignificant compared to the time it takes to run the inference process, even for the smallest models with the fastest inference