Technology

Well that description of a car is actually fairly close to the fundamentals, add an engine or motor and a steering wheel, and you've got it. Yes, a lot of engineering goes into the best possible realization of those basics, efficiency, suspension, safety, maintenance, and just a ton of more stuff, and it is a very valued execution above and beyond what, say, the Model T delivered. Automotive engineers have done hard and valuable work and complicated work, but no one is surprised that Model T led to faster, more comfortable, safer, more convenient vehicles that move around. It is a bit more surprising that LLM architecture works as well as it does while always focusing on the next token without ability to go further, best case running through and messing up and regenerating until you have operator appropriate output.

The 'seahorse emoji' was a pretty fun example of this at work, as it didn't have a seahorse emoji, but since it wasn't trying to generate the emoji, it started by building up the words to confirm and introduce the emoji since obviously there will be one, then putting up a wrong thing, then the words that would go after the wrong thing, but the weight still suggested there should be a correct answer and to start generating words for another try, and so on. "Reasoning" does the job of incurring this hit out of sight a lot of the time. Looking at the reasoning chains you'll see this behavior a fair amount, that the model suggests words that build toward an answer but fails on the key word and retries until something tests right or it models that it tried enough and it can't find the key word that would have been expected. It can of course digest it's own output and summarize the result without showing the operator spinning out, but it at all times is operating on the fundamental principle of model+very cleverly managed context influencing an answer one token at a time and ideally discarding the first run.