this post was submitted on 17 Nov 2023
1 points (100.0% liked)

LocalLLaMA

1 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago
MODERATORS
communick@poweruser.forum
1
What UI do you use and why? (alien.top)
submitted 10 months ago by Deadlibor@alien.top to c/localllama@poweruser.forum
57 comments fedilink hide all child comments
 

From the wiki:

Text generation web UI

llama.cpp

KoboldCpp

vLLM

MLC LLM

Text Generation Inference

you are viewing a single comment's thread
view the rest of the comments
[โ€“] CardAnarchist@alien.top 1 points 10 months ago (1 children)

I just switched to KoboldCpp from Text Geb UI 2 days ago.

The OpenAI extension wouldn't install for me and it was causing issues with SillyTavern which I use as a frontend.

I'm actually really happy now that I've switched.

KoboldCpp is so simple is great. I've written a simple batch file to launch both KoboldCpp and SillyTavern. All I have to do if I want to try a new model is edit the part of the batch pointing to the name of the model and it just works.

On top of that I can load more layers onto my GPU with KoboldCpp than Text Gen UI so I'm getting faster speeds.

[โ€“] IamFuckinTomato@alien.top 1 points 10 months ago

Have you tried installing the missing package files it shows when u tried installing the openai extension?
I had the same issue and installing those missing packages via the cmd_windows in the same folder.