Maybe it's overkill, idk, but if you want higher accuracy, it's an option
You can just list examples from your dataset and let the LLM complete the last one
Example:
# Classify text
(a) advertisement
(b) poetry
(c) information
Ignore real-time Al and customers will do the same to you. Our vector database is AI-ready and proven at scale.
Class: (a)
I find no peace, and all my war is done. I fear and hope. I burn and freeze like ice. I fly above the wind, yet can I not arise
Class: (b)
YOUR BEST COMES OUT OF THE BLUE. EXPLORE BOISE STATE
Class: (a)
Two-month ramp closure: northbound OR 99W onto OR 217 north Starts May 31 of Transportation Oregon Department OR 217 AUXILIARY LANES
Class: (c)
Staying healthy. Staying active. We have it all right here. IN YOUR PRIME LEARN MORE LIVING YOUR BEST LIFE
Class: (a)
Go further, FASTER. Take the world's premier English- proficiency test in less than 2 hours!
Class: (a)
A rhinoceros beetle is a living thing. Rhinoceros beetles grow and respond to their environment. They need food and water.
Class: (c)
Our vice runs beyond all that old men saw, And far authentically above our laws, And scorning virtues safe and golden mean, Sits uncontrolled upon the high extreme.
Class: (b)
{your text here}
Class: ({generate one token}
I don't know about the task you have in mind specifically, but you can do just about anything with a 13B llama model. Picking a fine-tune doesn't matter if you use examples instead of instructions. 7B Mistral seems to do fine with this example (even GPT2 can do some classification), but in-context learning is remarkably better at 13B, picking up a lot more nuance