LocalLLaMA

4 readers

4 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago

MODERATORS

submitted 1 year ago by Deadlibor@alien.top to c/localllama@poweruser.forum

57 comments fedilink hide all child comments

From the wiki:

you are viewing a single comment's thread
view the rest of the comments

[–] Flashy_Squirrel4745@alien.top 1 points 1 year ago

Text Generation webui for general chatting, and vLLM for processing large amount of data using LLM.

On an RTX3090 vLLM is 10~20x faster than llama.cpp for 13b awq models.