Found KoboldCpp on a guide somewhere and only used that. I barely even know much about anything else. I Just use GGUF and never worry about the so-called "VRAM"
cubestar362
joined 11 months ago
Found KoboldCpp on a guide somewhere and only used that. I barely even know much about anything else. I Just use GGUF and never worry about the so-called "VRAM"
7b models are much better than most 13b models at the moment. Maybe when Mistral 13b comes out you'll see more of them.