MoffKalast

joined 1 year ago
sorted by: new top controversial old
RWKV v5 7b, Fully Open-Source, 60% trained, approaching Mistral 7b in abilities or surpassing it. in c/localllama@poweruser.forum
[–] MoffKalast@alien.top 1 points 11 months ago (2 children)

Well it seems a lot better at Slovenian than LLamas or Mistral, especially for a 3B model, although it mostly just rambles about stuff that's vaguely related to the prompt and makes lots of grammatical mistakes. The 7B one ought to be interesting once it's done.

40x or more speedup by selecting important neurons in c/localllama@poweruser.forum
[–] MoffKalast@alien.top 1 points 11 months ago

I doubt it, most of their leverage is in being the only suppliers of hardware required for pretraining foundational models. This doesn't really change that.

40x or more speedup by selecting important neurons in c/localllama@poweruser.forum
[–] MoffKalast@alien.top 1 points 11 months ago (3 children)

Would be interesting to see if this can help speed up CPU inference with regular RAM, after all 128 GB of DDR5 only costs like $300 which is peanuts compared to trying to get any where close as much VRAM.

If it scales linearly then one could run a 100B model at the speed of a 3B one right now.

1
Chain of thought really helps :P (alien.top)