this post was submitted on 18 Jun 2026

89 points (97.8% liked)

Programming

27689 readers

519 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Follow the programming.dev instance rules
Keep content related to programming in some way
If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev

founded 3 years ago

MODERATORS

snowe@programming.dev

Ategon@programming.dev

UlrikHD@programming.dev

bugsmith@programming.dev

Spyro@programming.dev

Can gzip be a language model? (nathan.rs)

submitted 3 weeks ago by cm0002@mander.xyz to c/programming@programming.dev

17 comments fedilink hide all child comments

all 18 comments

sorted by: hot top controversial new old

[–] AnAmericanPotato@programming.dev 50 points 3 weeks ago (2 children)

The coolest, and often most confusing thing about computer science, information theory, and perhaps reality in general, is how everything becomes more or less equivalent if you boil it down and twist it around a little.

Everything is sorting. Everything is compression. Everything is geometry. Everything is language. Everything is music. Everything is, like, waves, man. *puff*

Or more accurately, everything can be expressed in any of those other things' terms.

These are not new ideas, but computers have made them provably and demonstrably true in many contexts, and I think that's super cool.

[–] partofthevoice@lemmy.zip 4 points 3 weeks ago* (last edited 3 weeks ago) (3 children)

Does it tell us something about reality if that’s true? I think it should… It reminds me of oneness.

[–] mumblerfish@lemmy.world 6 points 3 weeks ago

I wouldn't necessarily say it "tells us something about reality", as the expression goes, but it is useful in describing reality. Like the last statement of waves, which was supposed to be an spaced out exageration, is how much of physics is built. You look at something and wonder how to describe it. Sometimes it make sense to start with a single pulse, wave, or oscillator, which does not solve it completely and so you add more perturbations to it. You do this sort of stuff basically everywhere in physics. Everywhere else, some other correspondence usually appear. In computer science you use hard problems to design cryptos, physics gets stuck at the same problems. String theory uses algebraic geometry and end up with models where the areas they cannot solve is where elliptic curve crypos come from.

[–] hirihit640@sh.itjust.works 4 points 3 weeks ago

that's probably generalized too far and reaches into pseudo-science.

Just understand the concept of Turing-completeness, and the idea that many systems are Turing equivalent.

[–] MonkderVierte@lemmy.zip 4 points 3 weeks ago (2 children)

The "everything is waves" part, yes.

[–] partofthevoice@lemmy.zip 4 points 3 weeks ago (2 children)

But can everything be waves? Waves need to propagate through a substrate… so if everything is a wave, what is space?

[–] AnAmericanPotato@programming.dev 5 points 3 weeks ago (1 children)

Now we're getting into linguistics with the question of "what is a wave?"

In quantum physics, basically everything is waves, in the sense that the same mathematical formulae used to describe waves are used to describe quantum phenomena. The intuitive human-scale dynamics of waves don't necessarily apply though.

For example, sound waves can't propagate through a vacuum, but light waves can. Aside from that, they follow mostly the same rules. You can use the same math the describe interference of sound waves and light waves, for example.

People talk about the "particle/wave duality" of photons because in some ways they behave like waves and in some ways they behave like particles. But both of those words are stretched a little from their everyday plain-english usage, and the precise reality would require years of study to understand.

Plain English wasn't made to be that precise or objective. That's why we use math. :)

I'm no expert in quantum physics so take this all with a grain of salt.

[–] partofthevoice@lemmy.zip 4 points 3 weeks ago* (last edited 3 weeks ago) (2 children)

Thanks for the thorough reply!

What I’m gathering is that “wave” can refer to a behavioral pattern that is substrate independent — it refers to a logical function more than it does an ontological presence. That said, quantum waves are a substrate that exists beneath the material manifestations you and I experience (called a “wave” more-or-less for its mathematical properties)?

If that’s fair, would it be correct to call the quantum wave a “substrate” as I did?

and you know another thing about quantum field theory I don’t quite understand… I think it still depends on a four dimensional backdrop universe, for these fields to pervade. That fourth dimension is time, which is function of entropy. If time exists, that means the backdrop isn’t static — it evolves. That means it needs a fundamental explanation as well, something more than being just a background. No?

[–] AnAmericanPotato@programming.dev 3 points 3 weeks ago

What I’m gathering is that “wave” can refer to a behavioral pattern that is substrate independent — it refers to a logical function more than it does an ontological presence

I think that's a good way of putting it.

As for what counts as a "substrate", I have no idea! In the old days, the idea of a substance that permeated seemingly-empty space was common. See: https://en.wikipedia.org/wiki/Aether_theories

Nowadays, the idea of aether has been discarded for the most part. But that said, there's still plenty we don't understand, like dark matter. There's no consensus on what dark matter is exactly; there are many competing theories. What we know is that there are observable phenomena that can't be explained without something that acts (roughly, at least) like matter in terms of its effect on gravity, but doesn't interact with electromagnetism like normal matter. That "something" is called dark matter, but its fundamental nature is an open question.

[–] supersquirrel@sopuli.xyz 2 points 2 weeks ago* (last edited 2 weeks ago)

Subatomic particles are waves of probability.

It is worth looking up the Wave Equation and meditating on the fact that waves are solutions to a problem/set of conditions around conservation of energy.

It is an open ended definition not one that points out a discrete thing.

Surface waves for example such as Rayleigh Waves and Love Waves are solutions to the conservation of energy of a wave that cannot propagate past a free surface and thus energy in that direction must be conserved some other way through the solution of a surface wave.

https://visualpde.com/basic-pdes/wave-equation.html

https://visualpde.com/sim/?preset=waveEquation

^this is really fun I found it by accident because of this post.

https://en.wikipedia.org/wiki/Wave_equation

The second derivative (the acceleration) of energy in terms of time t is equal to a constant multiplied by the second derivative of energy in terms of distance x...

It suggests a basic back and forth transformation or equivalence at the heart of it, a wave is a relation embodied within physical constraints.

[–] MonkderVierte@lemmy.zip 1 points 3 weeks ago* (last edited 3 weeks ago)

what is space?

Disturbed vacuum. By waves and matter (which is made of "condensed" waves).

[–] supersquirrel@sopuli.xyz 2 points 2 weeks ago* (last edited 2 weeks ago)

Everything isn't waves, rather things happen as a summation of all probabilities in a wavefunction unless bounded by imposing factors that force the phenomena of particles... meaning everything perhaps could behave as waves depending on the situation and details. It is the default background tendency until absolute limitations are imposed on a system through measurement or interaction.

[–] litchralee@sh.itjust.works 17 points 3 weeks ago (1 children)

For an explainer of the theory behind the "language modeling is compression" paper, this video by 3Blue1Brown is especially relevant: https://youtube.com/watch?v=l6DKRf-fAAM

[–] ghodawalaaman@programming.dev 2 points 3 weeks ago

its fun to watch 1b3b videos even though I dont understand anything. i feel smart watching his videos

[–] palordrolap@fedia.io 7 points 3 weeks ago

Now I want to see this for the other compression algorithms. If I had to pick just one: bzip2.

[–] phoenixz@lemmy.ca 2 points 3 weeks ago

Very interesting, thanks!

[–] supersquirrel@sopuli.xyz 1 points 2 weeks ago* (last edited 2 weeks ago)

This is precisely why LLMs and AI are 99% a scam.

AI is the minimum value of a well structured dataset obtained by a destructive, hallucinatory compressor that is MAXIMALLY inefficient with inefficiency increasing nonlinearly as a model gets bigger... while it does demonstrate power, this observation that a well structured, QC'd, properly curated dataset reveals a latent intelligence in good data clearly points to functional programming, relational programming and the profession of the librarian and archivist as the directions where the genesis point of intelligence can be pursued, not these bullshitting AI's which demonstrate a degraded truth in a stupendously hamfisted, wasteful way that sends amateurs looking hopelessly looking in the wrong direction.

In otherwords, the algorithm and model are worthless, costly junk, it is the well structured, large high quality dataset and the humans that maintain and contextualize it that are precious.

Scientists could have told computer people this was true a long time ago if they had listened.