Supercomputers once required large power plants to operate, and now we carry around computing devices in out pockets that are more powerful than those supercomputers.
There’s plenty of room to further shrink the computers, simplify the training sets, formalize and optimize the training algorithms, and add optimized layers to the AI compute systems and the I/O systems.
But at the end of the day, you can either simplify or throw lots of energy at a system when training.
Just look at how much time and energy goes into training a child… and it’s using a training system that’s been optimized over hundreds of thousands of years (and is still being tweaked).
AI as we see it today (as far as generative AI goes) is much simpler, just setting up and executing probability sieves with a fancy instruction parser to feed it its inputs. But it is using hardware that’s barely optimized at all for the task, and the task is far from the least optimal way to process data to determine an output.
Hardships not seen in generations. I saw the oil crisis in the 70s and my grandparents were born in the great depression, fought in WWII and went through food rations.
Seems like the boomers just got an unusual reprieve from hardships that have otherwise been seen by pretty much every generation.