this post was submitted on 21 Nov 2023
1 points (100.0% liked)

LocalLLaMA

1 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago
MODERATORS
communick@poweruser.forum
1
Should I use LoRa, RLHF or DPO? (alien.top)
submitted 10 months ago by hadal1337@alien.top to c/localllama@poweruser.forum
1 comments fedilink hide all child comments
 

I'm thinking of using Llama 2 to detect spam messages:

  1. The model will first be fine tuned using LoRa/PEFT with some public dataset.

  2. Then, when given a block of text, it will decide if it's spam and provide reasons for the user.

  3. However, there can be false positives etc., so I figured a way to combat this would be to let the user tell the model if the response is correct or wrong (thumbs up/down).

Based on my requirements, is it better to use RLHF or DPO? Am I over complicating this, will fine tuning it based on user feedback work too?

top 1 comments
sorted by: hot top controversial new old
[–] oKatanaa@alien.top 1 points 10 months ago

You're better off using something like BERT rather than shooting a pigeon with a ballistic missile. It easier, cheaper, faster and much more reliable.