212
this post was submitted on 11 Aug 2023
212 points (96.5% liked)
Technology
59377 readers
5324 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Even if this is true, which I doubt, why not edit your robots.txt to disallow them to index it and leave the content up?
Because then they are hosting content that actively won't show up in any search?
Hosting web content (especially with lots of images) costs money. Historically, sites like cnet were built around ad revenue. It is worth keeping a review of a 2001 GE Refrigerator online if it comes up when people search for the model number (generally to figure out specs to shop for a replacement). It is not worth it if that is showing up when people want to buy a 2023 GE Refrigerator or actively penalizes you for having too much "old" content and so forth.
And if it won't generate any ad revenue going forward? baleeted!