LocalLLaMA

1 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago

MODERATORS

communick@poweruser.forum

Memory Needed while fine tuning Llama2-7b model (alien.top)

submitted 10 months ago by Choice_Diver_2585@alien.top to c/localllama@poweruser.forum

0 comments fedilink hide all child comments

My longest context length is 4k. I want to fine tune Llama2-7b model on this dataset. How much RAM I will need if I load model with 4bit quantization using the bitsandbyte.

I faced with OutOfMemory Error with 24G RAM.

Thank you!

no comments (yet)

sorted by: hot top controversial new old

there doesn't seem to be anything here