Everything I have read about how LLM’s work suggest that you’re giving them too much credit. Their “thinking” is heavily based on studied examples to the point that they don’t seem capable of original “thought”.
For instance, there was a breakdown of the capabilities of some new imaging models the other day (one of the threads on DB0) that showed that none of the tested models were able to produce a cube balanced on a sphere because there were simply too few examples of a cubic object balancing on a spherical one in its learning model. When asked to show soldiers, the ones that could produce more accurate images could not produce accurate diversity because their improved rendering was due to it drawing from a more limited, and thus less creative, dataset. The result was that it kept looking like it had a specific soldier “in mind” rather than an understanding of soldiers in general.
These things would be trivial for even a child to do, though they may not be able to produce the “uncanny valley” effect that AI is good at. If a kid knows what a cube is, knows what a sphere is, and understands the request, they can easily draw a cube on a sphere without having seen an example of that specific thing before.
I agree that the parrot analogy isn’t correct, but neither is the idea that these things will learn from their own echo chamber in the way you have described. Maybe the idea of dreaming is more accurate—an unusual shuffling of input to make bizzaro results that don’t have any intrinsic meaning at all beyond their relation to the data that is being used.
@aihorde@lemmy.dbzer0.com draw for me Geordi La Forge drawing a mouse in the USS Enterprise conference room.