LocalLLaMA

3 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago

MODERATORS

communick@poweruser.forum

Why are you running local models? What are you doing with them? (alien.top)

submitted 1 year ago by ExtremelyQualified@alien.top to c/localllama@poweruser.forum

28 comments fedilink hide all child comments

I’m fascinated by the whole ecosystem popping up around llama and local LLMs. I’m also curious what everyone here is up to with the models they are running.

Why are you interested in running local models? What are you doing with them?

Secondarily, how are you running your models? Are you truly running them on a local hardware or on a cloud service?

you are viewing a single comment's thread
view the rest of the comments

[–] Cerus@alien.top 1 points 1 year ago

Gamified RP backend for an interface I'm making with the Godot game engine, running 13B EXL2 models on a 16GB 4060TI, hooked up via the streaming WS API in Ooba.

Out of all the silly hobbies I've poked around with, this has been by far the most interesting.