LocalLLaMA

1 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago

MODERATORS

communick@poweruser.forum

Why isn't anyone building an Oogabooga-like app for Android and iPhone? (alien.top)

submitted 10 months ago by Winter_Tension5432@alien.top to c/localllama@poweruser.forum

14 comments fedilink hide all child comments

With high-end Android phones now packing upwards of 24GB of RAM, I think there's huge potential for an app like this. It would be amazing to have something as powerful as the future Mistral 13B model running natively on smartphones!

You could interact with it privately without an internet connection. The convenience and capabilities would be incredible.

you are viewing a single comment's thread
view the rest of the comments

[–] _Lee_B_@alien.top 1 points 10 months ago

It's not just RAM, you also need the processing power. Phones can't do *good* LLMs yet.

If you watch the chatGPT voice chat mode closely on android, what it does is listen, with a local voice model (whisper.cpp), and then answers generally/quickly LOCALLY, for the first response/paragraph. While that's happening, it's sending what you asked to the servers, where the real text processing takes place. By the time your phone has run the simple local model and gotten a simple sentence for the first response and read that to you, it has MOSTLY gotten the full paragraphs of text back from the server and can read that. Even then, you still notice a slight delay.