this post was submitted on 16 Apr 2026
9 points (80.0% liked)

Programming

26563 readers
182 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

  • Follow the programming.dev instance rules
  • Keep content related to programming in some way
  • If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev

founded 2 years ago
MODERATORS
snowe@programming.dev
Ategon@programming.dev
UlrikHD@programming.dev
bugsmith@programming.dev
Spyro@programming.dev
9
Are we doing crimes when scrapping data online? For example public available music? (jeferson.me)
submitted 3 days ago by Shin@piefed.social to c/programming@programming.dev
34 comments fedilink hide all child comments
 

I'm trying to get to a reason on this, but my point reach to a limit.

I've the feels that scraping the internet for public accessible data, like for example open and public music on Spotify wouldn't be a crime, but the distribution would be. At the same token, this is seem as a crime, while Google does the same and nothing happens, even worse, if this get regulated, Google would have a huge advantage on anyone else.

So, my deeper question is: "Is copyright dead?"

top 34 comments
sorted by: hot top controversial new old
[–] alex_riv@lemmy.org 1 points 14 hours ago

the robots.txt angle is interesting. legally it's not binding but practically violating it can affect how courts view your conduct (vs. good faith compliance).

for public data that's already freely accessible with no login, the case law is actually more permissive than people think — especially in the US after the hiQ v. LinkedIn ruling. the EU is more restrictive.

the music data use case you mention: scraping metadata (song titles, artist names) is different from scraping audio. metadata is generally considered factual/non-copyrightable. audio is obviously protected.

personally i've built some music tools that aggregate public chord/tab data. the legal gray area is real but the risk profile is different depending on what you're doing with it.

[–] one_old_coder@piefed.social 17 points 3 days ago* (last edited 3 days ago) (1 children)

It's illegal but all AI companies do it more than you'll ever do. You have my permission.

I still buy on Bandcamp because they deserve it.

[–] TehPers@beehaw.org 1 points 2 days ago

It's illegal

Sauce? Also, where?

[–] esc@piefed.social 14 points 3 days ago (2 children)

I can't care less about copyright and 'crimes' of copying.

[–] thingsiplay@lemmy.ml 1 points 2 days ago

So you don't care about GPL and Open Source then?

[–] pinball_wizard@lemmy.zip 3 points 3 days ago* (last edited 3 days ago)

Legally, check your local laws or just be sure to cover your ass with tor or a VPN with an anonymous core.

Ethically, just obey Wheaton's Law: "Don't be a dick."

With web scraping, I can think of two ways Wheaton's law applies:

  1. Scrapers should blend into existing background web traffic. They should be slow enough to not overwhelm their servers. This requires babysitting any new scraper until one is sure it is tuned to be safe for the scraped site.

  2. Any scraped content shouldn't be re-hosted in a way that harms the original content creators. Sharing is lovely. Harming artists sucks. Finding the right balance between preservation and respect can take some thought, but it's usually actually a pretty wide road.

[–] thingsiplay@lemmy.ml 2 points 2 days ago (1 children)

As far as I understand, Google scrapes data, processes it and uses it for commercial cases. It's a company, not a private person scraping and using for personal cases. A very important distinction.

[–] Shin@piefed.social 1 points 2 days ago

Since it's a company, it should not use our data, right? right? It's my data, it can't use my post for training, right? It's not fair use... right?

[–] TehPers@beehaw.org 2 points 2 days ago* (last edited 2 days ago) (1 children)

There's no obvious answer to your question without more information (for example, where are you?) but I'm not aware of scraping being illegal anywhere, with some exceptions. For example, in the US (where I am), as long as you're not doing "illegal hacking" to scrape your data, you're probably fine.

There are TOSs that websites like to impose as well. If you have to agree to one to access any data, you should follow it. Breaking the TOS isn't really "illegal" in a criminal sense (in the US), but you may expose yourself to anything from being blocked from the site to a lawsuit. Bypassing blocks might also be illegal, though you'd have to speak to a lawyer to know more about that.

[–] Shin@piefed.social 1 points 2 days ago (1 children)

That's the point, my focus is on the "Europe" as a general place, since they need to sync the "law" to some degree, there is different levels, but the base line are the same.

Most public data, like all the music in Spotify don't require a cookie. So I could in theory scrape all the Spotify music to "listem later". This wouldn't be "illigal", but if that's the case Annas Archive should be "fine"... (I know that they are distributing, and this is the fight)

But, if they scrapped the music, and I scrape we would have the same "dataset", so if I download the Annas "dataset", would it be different from mine? So if I prefer to download the Anna's dataset instead of scrape myself, would this be illigal? They aren't selling (on the contrary of Google).

There is way to many questions in my head :(

[–] TehPers@beehaw.org 2 points 2 days ago

This wouldn't be "illigal", but if that's the case Annas Archive should be "fine"... (I know that they are distributing, and this is the fight)

I don't know much about European law, but redistribution changes things a lot here in the US. At least here, it then gets into copyright law, and you'd be reproducing copyrighted works without authorization (the Internet Archive attempted to get around this with books by getting legitimate copies of the books, digitizing them, then "lending" the digital copies of those books).

So if I prefer to download the Anna's dataset instead of scrape myself, would this be illigal?

No idea in Europe. In the US, it might be, depending on what the contents of the work are. I believe Anna's Archive would count as piracy in this case, though scraping directly from Spotify might not be because they are redistributing the music with authorization from the copyright holder. It gets pretty confusing, honestly.

Regardless, if you aren't doing things at large scale, even if you are breaking a law by downloading pirated content, it's unlikely anyone will care. People usually only really start caring if you start redistributing stuff, so as long as you aren't hosting what you're scraping, you're unlikely to run into any trouble.

[–] misk@piefed.social 5 points 3 days ago* (last edited 3 days ago)

Not only copyright is dead but so is licensing of things in general. This means there’ll be less original work from both commercial and non-commercial projects. Commercially there won’t be ways to profit so why bother. On the libre licensing front why would you contribute code to GPL licensed projects or release art under Creative Commons if it’s going to be license washed anyway?

[–] Luminous5481@anarchist.nexus 4 points 3 days ago

no, in almost all cases internet piracy is not a crime. it is a civil issue. now if you were scraping information that wasn't public, that could be a crime depending on the circumstances.

[–] phonics@lemmy.world 1 points 3 days ago

Google would have some kind of licence in place I suspect. But what about people with photographic memory?