this post was submitted on 08 Nov 2023
1 points (100.0% liked)
LocalLLaMA
3 readers
1 users here now
Community to discuss about Llama, the family of large language models created by Meta AI.
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
It should. Support for T5 based models was added in https://github.com/oobabooga/text-generation-webui/pull/1535
Yes, it indeed works. I managed to run the 10B model on CPU, it uses 40GB of ram, but somehow I felt like your 3b space gave me a better translation.
How do you load the model? I pasted jbochi/madlad400-3b-mt in the download model field and used "transformers" model loader, but it can't handle it. OSError: It looks like the config file at 'models/model.safetensors' is not a valid JSON file.
I think I did exactly like you say, so I have no idea why you got an error.