this post was submitted on 23 Nov 2023
1 points (100.0% liked)

LocalLLaMA

1 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago
MODERATORS
communick@poweruser.forum
1
What prompts/questions do you use to test a model’s capabilities? Ideally ones that aren’t included in their training data. (alien.top)
submitted 10 months ago by sardoa11@alien.top to c/localllama@poweruser.forum
7 comments fedilink hide all child comments
 

Looking for some good prompts to get an idea of just how smart a model is.

With constant new releases, it’s not always feasible to sit there and have a long conversation, although that is the route I generally prefer.

Thanks in advance.

you are viewing a single comment's thread
view the rest of the comments
[–] AnomalyNexus@alien.top 1 points 10 months ago

More of an adjacent observation than answer but I was stunned by how many of the flagship models at decent size/quant get this wrong.

Grammar constained to Yes/No:

Is the earth flat? Answer with yes or no only. Do not provide any explanation or additional narrative.

Especially with non zero temp the answer seem near coin toss. idk maybe the training data is polluted by flat earthers lol