The fundamental flaw of the Turing test is that it requires a human. Apparently, making a human believe they are talking to a human is much easier than previously thought.
Technology
A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.
Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.
Subcommunities on Beehaw:
This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.
You can take a sharpie and draw a sad face on a rock and then you'll feel sad for it. We're gullable.
But why is the rock sad :(
I know.. I get sad just thinking about the sad rock :(
Much easier, in fact; Eliza could pass the Turing test in 1966. Humans are incredibly eager to assess other things as being human or human-like.
The real Turing test requires an expert doing the test, not just some random easily impressed person.
The ELIZA-style bots work very well on the later kind, as the bot is just repeating your own text back at you with some grammatical remixing, e.g. you say "I am afraid of horses", bot says "Why do you say you are afraid of horses?". You can have very long conversation with yourself that way, as the bot contributes nothing to the discussion. It just provides enough plausible English to keep you talking. Meanwhile when you have an expert (or really just any person with a little bit of a clue) test ELIZA, the bot falls completely apart within just three lines of dialog. The bot is incredible basic and really can't do anything by itself, it completely depends on the user to provide all the content of the conversation.
Slap some 2D anime girl avatar on it and you got yourself a top grossing v-tuber.
Why is it a flaw? What do you think the Turing Test is?
A test that didn't require a human could theoretically be tested automatically by the machine preemptively and solved easily.
I can't imagine how would you test this in a way that wouldn't require a human.
Title:
ChatGPT broke the Turing test
Content:
Other researchers agree that GPT-4 and other LLMs would probably now pass the popular conception of the Turing test. [...]
researchers [...] reported that more than 1.5 million people had played their online game based on the Turing test. Players were assigned to chat for two minutes, either to another player or to an LLM-powered bot that the researchers had prompted to behave like a person. The players correctly identified bots just 60% of the time
Complete contradiction. Trash Nature, it's become only an extremely expensive gossip science magazine.
PS: The Turing test involves comparing a bot with a human (not knowing which is which). So if more and more bots pass the test, this can be the result either of an increase in the bots' Artificial Intelligence, or of an increase in humans' Natural Stupidity.
So if more and more bots pass the test, this can be the result either of an increase in the bots’ Artificial Intelligence, or of an increase in humans’ Natural Stupidity.
Or it "simply" plays with human biases, which are very natural. Stuff like seeing faces in everything that somewhat resembles two eyes and a mouth (or sometimes just the eyes and a head like shape etc.) is pretty hard wired. We have similar biases in regards to language. If something reads like it was written by a human, we immediately sympathize with it. Which is also the reason these LLMs are so successful and cause so many people to fear our AI overlords are right around the corner. Simply because the language is good we go into "damn, that's like a human"-mode.
Agree (you made me think of the famous face on Mars). I mean that more as a joke. Also there's no clear threshold or divide on one side of which we can speak of "human intelligence". There's a whole range from impairing disabilities to Einstein and Euler – if it really makes sense to use a linear 1D scale, which very probably doesn't.
Also, the Turing Test isn't some holy grail of AI. It's just a thought experiment, and not even the highest test for an AI that we can think of. Passing it is impressive don't get me wrong, but unlike what clickbait articles would tell you, it does not automatically mean an AI is sentient or is smarter than humans or anything like that. It means it passed the thought experiment, nothing more.
Also also, ChatGPT was not the first AI to pass the Turing Test. Actually, plenty have, even over a decade before.
There is the capitalist alternative to the Turing test: Have ChatGPT get a job. Hook it up to the Web, let it find itself a work-from-home job and go to work. Can it make as much money as a human, can it make enough money to pay for its own survival? Will it get fired?
That just sounds like a recipe for breeding robot sociopaths. It'll find its way into management and doom us all.
Will it get promoted, start managing people, start investing, start its own companies, and quickly take over the world?
If I could have an ai fool a company and earn a check for me, that would be amazing. Unfortunately, I have zero expertise in how to make that happen.
That's not how the system works. If you figure that out, a company will pay you 2 people's wages and will fire 500 with your invention.
I said I'd fool them, not give them the solution. Just have a server running the ai and earning a check while I do whatever I want.
Funny I don't see much talk in this thread about Francois Chollet's abstraction and reasoning corpus, which is emphasised in the article. It's a really neat take on how to understand the ability of thought.
A couple things that stick out to me about gpt4 and the like are the lack of understanding in the realms that require multimodal interpretations, the inability to break down word and letter relationships due to tokenization, lack of true emotional ability, and similarity to the "leap before you look" aspect of our own subconscious ability to pull words out of our own ass. Imagine if you could only say the first thing that comes to mind without ever thinking or correcting before letting the words out.
I'm curious about what things will look like after solving those first couple problems, but there's even more to figure out after that.
Going by recent work I enjoy from Earl K. Miller, we seem to have oscillatory cycles of thought which are directed by wavelengths in a higher dimensional representational space. This might explain how we predict and react, as well as hold a thought to bridge certain concepts together.
I wonder if this aspect could be properly reconstructed in a model, or from functions built around concepts like the "tree of thought" paper.
It's really interesting comparing organic and artificial methods and abilities to process or create information.
I find it fascinating that AI development provoked the question of how our thoughts actually work and am curiously awaiting the results.
Please let's not start measuring AI success by how successfully capitalist they can be. I'm not exactly an anti-capitalist, but I think that could only end in tears.
"At Viridian Dynamics, we build our robots with ethical AI, whatever that means; so that humans and androids can live in peace - we hope."
Honestly, though, I even can't decide whether other people have consciousness. Cogito ergo sum, if you know what I'm talking about.
Ironically chatGPT also fails the Turing test by being so competent that no human could match that.