this post was submitted on 22 Nov 2023
1 points (100.0% liked)
LocalLLaMA
3 readers
1 users here now
Community to discuss about Llama, the family of large language models created by Meta AI.
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Does this technique affect the required RAM-size for inference?
I don't think so (unfortunately). The model size doesn't change, only the way it is traversed.
Can this technique be combined with lora with a not so low rank? Lora increases the learning time (I heard) but this should be no problem then anymore :)