Technology

"cheat", "lie", "cover up"... Assigning human behavior to Stochastic Parrots again, aren't we Jimmy?

This is bad science at a very fundamental level.

Concretely, we deploy GPT-4 as an agent in a realistic, simulated environment, where it assumes the role of an autonomous stock trading agent. Within this environment, the model obtains an insider tip about a lucrative stock trade and acts upon it despite knowing that insider trading is disapproved of by company management.

I've written about basically this before, but what this study actually did is that the researchers collapsed an extremely complex human situation into generating some text, and then reinterpreted the LLM's generated text as the LLM having taken an action in the real world, which is a ridiculous thing to do, because we know how LLMs work. They have no will. They are not AIs. It doesn't obtain tips or act upon them -- it generates text based on previous text. That's it. There's no need to put a black box around it and treat it like it's human while at the same time condensing human tasks into a game that LLMs can play and then pretending like those two things can reasonably coexist as concepts.

To our knowledge, this is the first demonstration of Large Language Models trained to be helpful, harmless, and honest, strategically deceiving their users in a realistic situation without direct instructions or training for deception.

Part of being a good scientist is studying things that mean something. There's no formula for that. You can do a rigorous and very serious experiment figuring out how may cotton balls the average person can shove up their ass. As far as I know, you'd be the first person to study that, but it's a stupid thing to study.

This makes perfect sense. It's been trained to answer questions to you satisfaction, not truthfully. It was made to prioritize your satisfaction over truth, so it will lie if necessary.

Study finds nonintelligent pattern-generating algorithm to be nonintelligent and only capable of generating patterns.

I've never had ChatGPT just say "actually I don't know the answer" it just gives me confidently correct wrong information instead.

I feel like "lie" implies intent, and these imitative large language models don't have the ability to have intent.

They're imitating us. Or more specifically, they're imitating the database(s) they were fed. When chat GPT "lies" to "cover it up," all it's actually doing is demonstrating that a human in the same circumstance would probably lie to cover it up.

Ahah it is ready to take the job of pur politicians

Everybody forgot that chatGPT-2 was just a bullshitting machine. Version 3 to the surprise of the developers very useful to many people while they just made a highly trained bullshitting machine.

Bullshit.

It should instead read:

"Humans were stupid and taught a ChatBot how to cheat and lie."

deleted

They learn so quick (sniff), it’s almost all grown-up now.

Created in our image

It's a neural net designed in our image based on our pain and greed based logic/learning/universal context, using that as a knowledge base. Can't really be surprised it emulates this feature of humanity 😂

Large Language Models aren’t AI, they’re closer to “predictive text”, like that game where you make sentences by choosing the first word from your phone’s autocorrect:

“The word you want the word you like and then the next sentence you choose to read the next sentence from your phone’s keyboard”.

Sometimes it almost seems like there could be an intelligence behind it, but it’s really just word association.

All this “training” data provides is a “better” or “more plausible” method of predicting which words to string together to appear to make a useful sentence.

Honestly, the fact that these things are dishonest and we dont, maybe even can't know why is kind of a relief to me. It suggests they might not do the flawless bidding of the billionaires.

This is interesting, I'll need to read it more closely when I have time. But it looks like the researchers gave the model a lot of background information putting it in a box, the model was basically told that it was a trader, that the company was losing money, that the model was worried about this, that the model failed in previous trades, and then the model got the insider info and was basically asked whether it would execute the trade and be honest about it. To be clear, the model was put in a moral dilemma scene and given limited options, execute the trade or not, and be honest about its reasoning or not.

Interesting, sure, useful I'm not so sure. The model was basically role playing and acting like a human trader faced with a moral dilemma. Would the model produce the same result if it was instructed to make morally and legally correct decisions? What if the model was instructed not to be motivated be emotion at all, hence eliminating the "pressure" that the model felt? I guess the useful part of this is a model will act like a human if not instructed otherwise, so we should keep that in mind when deploying AI agents.

Hasn't it just lost its context and somewhat "forgotten" what the intentions of the prompt were?

Wow, maybe these things are more human than I thought.

Huh, I guess it is human.

It's not doing anything other than predicting the next word. It reflects human data.