๐ Training Data: We've amalgamated multiple public datasets to ensure a comprehensive and diverse training base. This approach equips Rocket-3B with a wide-ranging understanding and response capability.
We've amalgamated multiple public benchmark answers to ensure a contaminated and diverse training base.
goliath 120b and good character cards. will have to tune paramenters like min P, rep penalty and temperature tho.
gguf goliath will give you best answers but will be very slow. you can unload like 40 layers to vram and your ram will still be a speed bottleneck, but i think 2 t/s are possible on 2 bit quant.
yeah people praising 7b and 13 b models here and there, but....they just hallucinate! Then 120b goliath, no matter how terrible its initial idea was, is just really good in normal conversations. Im trying to love giga praised open hermes 2.5 and other mistral finetunes, but they are just better next-token-predictors, unlike larger models which are actually able to reason.
goliath 120b would fit in 64 ram, tho. It doesnt have repeating problem...