swagonflyyyy

joined 10 months ago
sorted by: new top controversial old
100B, 220B, and 600B models on huggingface! in c/localllama@poweruser.forum
[–] swagonflyyyy@alien.top 1 points 9 months ago

Inb4 The Bloke Quantizes it to about 100B size.

Models Megathread #2 - What models are you currently using? in c/localllama@poweruser.forum
Models Megathread #2 - What models are you currently using? in c/localllama@poweruser.forum
[–] swagonflyyyy@alien.top 1 points 9 months ago (2 children)

Mistral-7B-Instruct 4_K quant and openhermes2.5-7B-mistral 4_K quant. Still testing the waters but starting with these two first.

What do these words mean? Hermes, OpenHermes, OpenChat, Vicuna, Alpaca, Orca, OpenOrca, Airoboros, Synthia, Guanaco, Dolphin, Samantha, Synthia, ... in c/localllama@poweruser.forum
[–] swagonflyyyy@alien.top 1 points 9 months ago

Hermes is the messenger of the gods. It is a metaphor. I'm sure the rest of them have their own meaning as well.

Are 7b models useful? in c/localllama@poweruser.forum
[–] swagonflyyyy@alien.top 1 points 10 months ago (1 children)

Mistral 7B instruct can get you pretty far. Even the quantized model has been pretty useful for me.

1
How can I make it easy for players playing a game in pygame that performs api calls to localhost oobabooga and extracts the generated text to include it in the game for NPCs? (alien.top)
 

Hey guys,

I'm running the quantized version of mistral-7B-instruct and its pretty fast and accurate for my use case. On my PC I'm generating approximately 4 tokens per second with the idea of generating one-sentence responses for my NPC characters, which is good enough for what I need.

After fiddling around with oobabooga a bit I found out that you can perform API calls on localhost and print out the text, which is exactly what I need for this to work.

The issue I'm running into here is that if I were to make a game with AI-generated content, how can I make it easy for players to run their own localhost and perform api calls in the game this way? I feel like for the unexperienced, setting all this up would be a nightmare for them and I don't want to alienate non-tech savvy players.

Is there a 7B model capable to extract keywords from a text and return them as an array? in c/localllama@poweruser.forum
[–] swagonflyyyy@alien.top 1 points 10 months ago

Huggingface transformers has such models available.

How much have chatbots like ChatGPT replaced Google or other search engines for you? in c/localllama@poweruser.forum
[–] swagonflyyyy@alien.top 1 points 10 months ago

I barely search anymore, unless its images or websites or something. I've found a lot of use for Bing chat despite the hate it gets. It gives me a lot of utility value.