Talk about publishing, everyone and their mom creating blogs and posting them everywhere posoble.. And these people just read quick start page of any new library and flood the internet with mediocre content. I'm tired of looking through hundreds of such article to find one whenever I want to do something which is just one step more than hello world.
vicks9880
joined 11 months ago
I have 50GB free, but I believe they reduced it to 20GB for new users. Plenty enough for the application data.
Vllm is performing good so far. Better than expected. Using distributed gpu and trying to work on extending gpu based on load. Need to figure out correct metric on which to trigger the scaling up/down
These are just names, LLaMA originally meant ( Large language model Meta AI), but it appears that its also the name of South American animal, thus creative people of internet who download those weights, fine-tuned it and published it with the other animals of same family like alpaca, vicuna, dalai (llama) etc.
There are more important information in these model names, which are suffix, like parameter counts13B, quantization methods GGUF/GGML, fine-tuning techniques like LoRA, fine-tuning parameters like Q6_K_M etc are used.