this post was submitted on 18 Nov 2023
1 points (100.0% liked)

LocalLLaMA

3 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago
MODERATORS
communick@poweruser.forum
1
Another test of logical ability for LLMs? (alien.top)
submitted 11 months ago by laca_komputilulo@alien.top to c/localllama@poweruser.forum
14 comments fedilink hide all child comments
 

Found this in a children's book of riddles:

Six brothers were spending their time together.

The first brother was reading a book.
The second brother was playing chess.
The third brother was solving a crossword.
The fourth brother was watering the lawn.
The fifth brother was drawing a picture.

Question: what was the sixth brother doing?

I cant get ChatGPT to answer correctly with the usual tricks, even after hinting to consider one and two-person activities and emphasizing the word "together".

After a bunch of CoT turns we arrive to a conclusion that this is an open ended question and not a riddle :)

After trying 3 times with fresh promots, I got a correct response once, but when prompted to provide supporting reasoning the model backtracked and started apologizing.

Cant test gpt 4 r/n...

you are viewing a single comment's thread
view the rest of the comments
[–] MeMyself_And_Whateva@alien.top 1 points 11 months ago

I got this deep, but wrong reply from Xwin-LM-70b-v0.1.Q5_K_S:

The sixth brother was most likely engaged in a activity that showcased his intellectual prowess or creativity, as were his other brothers. Since you have not provided any further information about their interests or surroundings, I cannot give a specific answer. However, it is reasonable to assume that the sixth brother was involved in an activity that stimulated his mind and demonstrated his unique talents.

Possible activities for the sixth brother could include:

  1. Playing a musical instrument, like the piano or violin.
  2. Writing a story or poem, expressing his creativity through literature.
  3. Solving mathematical puzzles or problems, showcasing his logical and analytical skills.
  4. Engaging in a debate or intellectual discussion with friends or family members.
  5. Conducting an experiment or working on an inventive project, similar to my own passion for innovation.

In summary, without more information, it is difficult to pinpoint the exact activity of the sixth brother. However, based on the context, it can be inferred that he was likely involved in a mentally engaging or creative pursuit.