nderstand2grow

joined 10 months ago
sorted by: new top controversial old
RWKV v5 7b, Fully Open-Source, 60% trained, approaching Mistral 7b in abilities or surpassing it. in c/localllama@poweruser.forum
[–] nderstand2grow@alien.top 1 points 9 months ago

your comment is so insightful, thank you. if there are resources I can read/watch to learn about this stuff, I'd be happy if you could share them.

1
What do these words mean? Hermes, OpenHermes, OpenChat, Vicuna, Alpaca, Orca, OpenOrca, Airoboros, Synthia, Guanaco, Dolphin, Samantha, Synthia, ... (alien.top)
 

I'm confused by all these prefixes that appear in the finetunes of base models. Is there a glossary of all these words and similar ones?

1
Storing LLM models on external SSD: Is the SSD speed important? Samsung T7 w/ USB 3.2 (read: 1050MB/s) vs. Fantom w/ Thunderbolt 3/4 (read: 2800MB/s) (alien.top)
 

(title)

A fun day evaluating LLM Chat GUIs/Servers in Docker. Here's what I learned... in c/localllama@poweruser.forum
[–] nderstand2grow@alien.top 1 points 10 months ago

I have ollama on my Mac (not Docker) and installed the ollama web UI. It works fine but their instruction on running ollama in a LAN network doesn't work for me. The flags they mention to add the CLI command throw an error (esp. the * part).

For roleplay purposes, Goliath-120b is absolutely thrilling me in c/localllama@poweruser.forum
[–] nderstand2grow@alien.top 1 points 10 months ago

You could use M2 Ultra instead ($6500) vs. 2x$15,000+rest

PSA: GBNF exists. Use it. in c/localllama@poweruser.forum
[–] nderstand2grow@alien.top 1 points 10 months ago (1 children)

I think this is only available on llama.cpp. I've been using it for a while for simple structured outputs and am extremely happy with the results. With OpenAI's function calling, I always had to write validators -- first to make sure the output is indeed a JSON, and then another validator to make sure the JSON complies with my JSON schema. grammar makes all of that redundant because it is 100% guaranteed to generate the desired output (including JSON).