this post was submitted on 23 Nov 2023
1 points (100.0% liked)

LocalLLaMA

3 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago
MODERATORS
communick@poweruser.forum
1
What prompts/questions do you use to test a model’s capabilities? Ideally ones that aren’t included in their training data. (alien.top)
submitted 11 months ago by sardoa11@alien.top to c/localllama@poweruser.forum
7 comments fedilink hide all child comments
 

Looking for some good prompts to get an idea of just how smart a model is.

With constant new releases, it’s not always feasible to sit there and have a long conversation, although that is the route I generally prefer.

Thanks in advance.

you are viewing a single comment's thread
view the rest of the comments
[–] tgredditfc@alien.top 1 points 11 months ago (2 children)

“Write the snake game using pygame”

[–] AdOne8437@alien.top 1 points 11 months ago

I fear this one will be part of the training/finetune data very soon.

[–] kpodkanowicz@alien.top 1 points 11 months ago

started asking it as well - seems to be very hard for 34b models to get it fully right @1