242
LLMs are eroding my software engineering career and I don't know what to do
(human-in-the-loop.bearblog.dev)
this post was submitted on 09 Jun 2026
242 points (98.4% liked)
Programming
27257 readers
264 users here now
Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!
Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.
Hope you enjoy the instance!
Rules
Rules
- Follow the programming.dev instance rules
- Keep content related to programming in some way
- If you're posting long videos try to add in some form of tldr for those who don't want to watch videos
Wormhole
Follow the wormhole through a path of communities !webdev@programming.dev
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I mean, AGI is inevitable, but it's never gonna come from these dinguses. They can't even look past LLMs far enough to pursue text diffusion.
To imagine we cannot possibly build a mind, or that it cannot possibly improve that same effort, is baffling. It changes the shape of the universe.
Just because it's possible does not mean it's inevitable. It's incredibly optimistic to think that we can get our shit together enough to pull it off before we destroy all our productive capacity through hubris.
Nothing's inevitable. And as for "building a mind", while it depends on precisely what you mean by "mind", it's totally possible that only a biological brain can produce minds as we understand the word "mind". Building AGI doesn't necessarily mean building a mind. And since thoughts seem to be properties of "matter", and there seem to be rules about which configurations of matter produce mind, we don't necessarily know that there are other configurations that can produce minds. We might produce something else equally interesting which still is not a mind.
Bollocks. Thought is a process, like math. Nothing meat does with signals is impossible in other substrates.
At the utmost extreme: surely we can simulate physics at whatever level is necessary for virtual brains to function. Physical neurons are not gonna rely on quantum chromodynamics. Mere chemistry will probably suffice.
And hand-waving things that are like-minds-but sounds like Chinese Room nonsense.
You're making a baseless assumption about the inner being of every process. If you simulate physics then you're actually doing different physics, where the map is not the terrain. If the hardware is different then the inner being of the thing may very well be different.
You're actually displaying a lack of imagination here. You're not considering things other than consciousness. If you simulate the processes which on the surface resemble the processes that you see in the brain when observing from the outside, what you produce may be something equally interesting and yet totally different in-itself from subjective experience.
You don't know as much as you think you know.
Woo-woo nonsense. Indefensible without invoking the supernatural.
Everyone you've ever met is a meat robot, and they run on physics. How they feel about it does not change that.
And I already said, we invented p-zombies. LLMs display intelligence without consciousness. Your hand-wavy what-if doesn't work as a gotcha because we're already there. Nonetheless - these shortfalls are a big fucking hint that magic isn't real.
that's precisely what I'm saying
That's precisely what I'm saying
Nobody said anything about a gotcha
Nobody said anything about magic.
If consciousness is a physical process, then a different physical process (such as an intelligent process running on different hardware) cannot be assumed to produce the same result (the result of conscious experience).
Like planes don't experience flight unless they flap.
This is stupid. I acknowledge that's not an airtight logical counterargument, but just, come the fuck on. You are asserting that neurons made of silicon, with identical observable function, wouldn't count somehow. Charitably: wouldn't work, somehow. That at least distinguishes it from standard Chinese Room horseshit. But if we can fake every neuron to do the same thing, or simulate the entire physical environment to do the same thing, of fucking course it's going to do the same thing. If the laws of the universe somehow mean only meat can experience being a true Scotsman, we can fake those laws.
You've picked a philosophical nit that is somehow at odds with Turing completeness. Unless you think physics are incomputable - it cannot matter what substrate they run on. It's literally math.
Are you claiming that planes experience flight?
k
k
Two problems with this: (1) The virtual neural net of LLMs don't have neurons made of silicon. Their neurons are virtual, abstract, not physical phenomena. (2) Even if we move to the idea of a positronic brain like Data from Star Trek or the Terminator, it still isn't our chemical-electrical brain which has different physical properties. This is very simple. It is a different physical object. It is different. It is not what we are.
k
If you are simulating it, it is a different thing.
This is a metaphysical assumption much closer to the "woo" that you keep accusing me of, and cursing at me about.
This contradicts your statement that "It's literally math" because you can calculate the difference between substrates.
No.
If a simulated person experiences consciousness, that experience is real. That's qualia, numbnuts. That is a mind.
I'm not gonna pick apart the word salad of 'calculating the difference between substrates' if you can't even keep your philosophy straight.
k
You're just throwing the assumption of experience into the sentence for no reason. You have beliefs about this stuff, not facts.
Do you know what if means?
You want to make this a matter of philosophy, and then you suck at philosophy. Hey buddy, do you have facts about other real people's experiences, or do you just have beliefs? Could you even demonstrate your own conscious experience to me?
And all of this is such tired Philosophy 101 crap, just so you can cling to 'aha but what if,' even though I have a concrete answer for what-if. Are we ruling out magic? Great, then physics can be simulated and a computer can host a mind that way. Its experiences would be identical to any free-range meatbag. If it wrote a book, you could read it. That would be real art. So in what fucking manner is its consciousness not real experience?
Hello.
When you said, "If a simulated person experiences consciousness, that experience is real", you're just saying "If my statement is true then my statement is true." You offer no basis for assuming that "a simulated person experiences consciousness" in the first place. You are simply assuming it. Your whole side of this entire conversation is just an assumption.
Yes and the simulation is a different physical process than the process that it's simulating.
'If it happens, it counts' is not a tautology when you insist happening and counting are different things. To the guy in the simulation, any experience is real. His consciousness entails all the processes you insist must be accounted for. It works the exact same way as it does in real life with real meat.
If you would insist 'well that's only simulating consciousness' - that counts.
Again, you have no basis for claiming that there is experience in the simulation in the first place. If you set up some simulation to mimic human behavior there's no reason to assume that it is experiencing that behavior. You're putting an assumption here.
I never insisted that it's "only simulating consciousness." I'm rejecting the baseless assertion that consciousness is what's being simulated. If you simulate the physical processes that we associate with consciousness (neural networks), even if it produces enough observable behaviors that we find the model useful, it is still a different physical thing and therefore we are unable to assert that it is experiencing consciousness.
Simulating physics from first principles is not "mimicking human behavior." Don't dismissively phrase it like a chatbot. If you insist the exact molecular exchanges in human neurons are a mandatory component, you could observe every subatomic event, not just the fact they talk to you like any other meatbag on the street.
You just did! Again! You think an entire simulated human being, that acts exactly like a living person for the same underlying reasons, must be different - somehow. Your consciousness only arises from the laws of physics and the shape of matter, but if we simulated both of those exactly, and got indistinguishable results, then nuh-uh.
This is dualism. This is Descartes torturing dogs and insisting they only act like they feel pain, unlike us real humans, because we're different and special. It's straight-up Chinese Room horseshit, where no demonstrable evidence of conscious thought is enough, unless it fits your preconceived notions of what minds look like.
Our "first principles" are knowledge, not a new universe. You're not absolutely re-creating the entirety of reality. You're making a model that is useful, something that is "good enough" that it will produce outputs that can be used to roughly predict the outcome of real processes.
No matter how detailed your simulation is, it's still physically different. You're admitting this every time you call it a simulation. You can tell the difference between a simulation and the thing it's simulating.
No, because I'm saying that consciousness is a physical process. You're saying that it's a mathematical process and that the substrate doesn't matter. I'm saying the substrate is a physical thing, and that consciousness is a physical processes, and so different substrates enable different physical processes. I don't claim that consciousness comes from some Beyond, or from Heaven, or from God. I'm saying it's a physical process, and that a simulation is a different physical process, no matter how detailed.
The premise is - you fucking can't. It acts ex-act-ly like the real thing, and for all the same reasons. Given that unreasonably high standard, you still insist, nuh uh. Would never be conscious. We're beyond 'but what if.' You're explicitly arguing that machine consciousness would not count. That any difference from exactly how humans work cannot be a mind. Fuck off. That's just novel bigotry. Dog-torturing prejudice.
If you can have a long-ass argument with someone and go away figuring they're a person like you - that's consciousness. That's the only way you can judge the inner life of any person you have ever met. And you want to pretend that someone meeting that standard, while you observe their brain at a subatomic level, is disqualified? Come back and fuck off again.
You cannot laud the perfect exactitude of... squishy biology and quantum foam... and still say nuh-uh when a whole-ass person arises from those exact processes. If you mean anything when you say it's all just physics, then whatever physics are required for consciousness, can be faked. The resulting person really does think, as surely as an emulated calculator does real math. It's not simulating math. It's doing math. The device is simulated, but the answers are real. Get it?
You literally made the simulation, so yes you can tell the difference. And the math is inevitably different, because the math of the simulation includes the math which defines the different substrate. So it is physically different, a different thing. Assume whatever you want, but in the end it is a physically different thing, and it takes different math to fully describe it. We only know for sure that conscious experience happens with this substrate.
k
k
good bye
2+2 on an emulated calculator gets a different answer than 2+2 on a real one, because there's different math. Math being physics in this case but not when it undercuts your whole point. Conscious experience must only be possible in this substrate, despite zero evidence to the contrary... and faking the entire substrate with physics math doesn't count because numbers work differently in rocks than in meat.
k.
If telling you to shut up is redundant then good riddance.
You say that, and GAs were used decades ago to design FPGAs to a spec. The evolved design worked perfectly on the test chip, so the design was copied onto a second chip and it failed. The logic gates were identical but the GA had utilised microscopic differences in the substrate and there were large areas of programmed chip totally unconnected to the main circuit. Without them, the first chip didn’t work any more.
There are likely quantum effects available at the size / scale of neurons, and it’s brave to say evolution wouldn’t exploit them if there was some benefit.
Yeah yeah yeah, probably exploiting capacitance instead of on-spec functionality, I'm well familiar with this example. It's not relevant - there's eight billion human brains in the world, and they generally still function despite the wild shit we put them through. They are not fragile.
A human mind is not balanced on a knife-edge, where one tiny difference breaks everything. They're complex enough that sometimes blowing a railroad spike clean through just alters functionality. It's still a mind. Subatomic interactions surely cannot be crucial here.
And again, this is only the extreme example. Y'think all known laws of the universe are mandatory? Great, simulate those too. Same answer: meat has no monopoly on thought because metal can fake the meat. There is no philosophical basis for even suggesting AGI is impossible, unless you start talking about souls.
The scale of neurons is too big for quantum effects, but that's contemporary understanding that may change in the future. We're really far from understanding both what mind is and how to make one