this post was submitted on 01 Nov 2023
1 points (100.0% liked)

LocalLLaMA

3 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago
MODERATORS
 

Was wondering if anyone had guide or some code to get started on Lora training. I want to get started on creating a customized Llama2 model with a small dataset that I created. Was wondering if anyone had a good starting point.

top 2 comments
sorted by: hot top controversial new old
[โ€“] samsekhar@alien.top 1 points 1 year ago
[โ€“] FPham@alien.top 1 points 1 year ago

One of these days I'll write 400 pages book "Lora and you, the love at first sight."

I trained more than 500+ LORAs using all methods possible, devised my own process of creating datasets, hybrid datasets, reverse datasets, while also writing my own training and chunking code (Training PRO extension for WebUI) and the benefit is that basically every single one I churn now is a raging success. At least in my own head.

But it is very hard to explain it without those 400 pages I didn't write.