LocalLLaMA

3 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago

MODERATORS

communick@poweruser.forum

Open-source LoRa training guide & Code? (alien.top)

submitted 1 year ago by Danny_Davitoe@alien.top to c/localllama@poweruser.forum

2 comments fedilink hide all child comments

Was wondering if anyone had guide or some code to get started on Lora training. I want to get started on creating a customized Llama2 model with a small dataset that I created. Was wondering if anyone had a good starting point.

you are viewing a single comment's thread
view the rest of the comments

[–] FPham@alien.top 1 points 1 year ago

One of these days I'll write 400 pages book "Lora and you, the love at first sight."

I trained more than 500+ LORAs using all methods possible, devised my own process of creating datasets, hybrid datasets, reverse datasets, while also writing my own training and chunking code (Training PRO extension for WebUI) and the benefit is that basically every single one I churn now is a raging success. At least in my own head.

But it is very hard to explain it without those 400 pages I didn't write.