This reads like OpenAI's fanfic on what happened, retconning decisions they didn't make, things they didn't (couldn't!) do, and thought that didn't occur to them. All indicating that the possibility to be infinitely better is not only possible, but is right there for their taking.
For the one in April, engineers created many new versions of GPT-4o — all with slightly different recipes to make it better at science, coding and fuzzier traits, like intuition.
Citation needed.
OpenAI did not already have this test. An OpenAI competitor, Anthropic, the maker of Claude, had developed an evaluation for sycophancy
This reality does not exist: Claude is trying to lick my ass clean every time I ask it a simple question, and while sycophantic language can be toned down, the behavior of coming up with a believable positive answer for whatever the user has, is the foundational core of LLMs.
“We wanted to make sure the changes we shipped were endorsed by mental health experts,” Mr. Heidecke said.
As soon as they found experts who were willing to say something else than "don't make a chatbot". They now have a sycophantically motivated system with an ever growing list of sticky notes on its desk: "if sleep deprivation then alarm", "if suicide then alarm", "if ending life then alarm", "if stop living then alarm", hoping to have enough to catch the most obvious attempts.
The same M.I.T. lab that did the earlier study with OpenAI also found that the new model was significantly improved during conversations mimicking mental health crises.
The study was basically rigged: it used 18 known and identified crises chat logs from ChatGPT - meaning the set of stuff OpenAI just had hard coded "plz alarm" for, and thousands of "simulated mental health crises" generated by FUCKING LLMs meaning they only test if ChatGPT can identify mental health problems in texts where it had written its own understanding of what meantal health crisis looked like. For fucks sake of course it did perfectly in guessing its own card.
TLDR; bullshit damage control