Selfhosted

It's a shame we don't have those banner ad schemes anymore. Cybersquatting could be a viable income stream if you could convince the cleaners to click banner ads for a faction of a penny each.

That's insane... Can't a website owner require bots (at least those who are identifying themselves as such) to prove at least they're affiliated with a certain domain?

I don't know what "12,181+181" means (edit: thanks @Thunraz@feddit.org, see Edit 1) but absolutely not 1.2181 × 10^185^. That many requests can't be made within the 39 × 10^9^ bytes of bandwidth − in fact, they exceed the number of atoms on Earth times its age in microseconds (that's close to 10^70^). Also, "0+57" in another row would be dubious exponential notation, the exponent should be 0 (or omitted) if the mantissa (and thus the value represented) is 0.

AI bots killing the internet again? You don't say

I had to pull an all nighter to fix some unoptimized query because I had just launched a new website with barely any visitors and hadn’t implemented caching yet for something that I thought no one uses anyway, but a bot found it and broke my entire DB through hitting the endpoint again and again until nothing worked anymore

fracking clankers.

You can also use crowdsec on your server to stop similar BS. They use a community based blacklist. You choose what you want to block. Check it out.

https://github.com/crowdsecurity/crowdsec

Fucking hell.

Yeah and that's why people are using cloudflare so much.

AI scrapers are the new internet DDoS.

Might want to throw something Infront of your blog to ward them off like Anubis or a Tarpit.

I run an ecommerce site and lately they've latched onto one very specific product with attempts to hammer its page and any of those branching from it for no readily identifiable reason, at the rate of several hundred times every second. I found out pretty quickly, because suddenly our view stats for that page in particular rocketed into the millions.

I had to insert a little script to IP ban these fuckers, which kicks in if I see a malformed user agent string or if you try to hit this page specifically more than 100 times. Through this I discovered that the requests are coming from hundreds of thousands of individual random IP addresses, many of which are located in Singapore, Brazil, and India, and mostly resolve down into those owned by local ISPs and cell phone carriers.

Of course they ignore your robots.txt as well. This smells like some kind of botnet thing to me.

Downloading you wallpapers? Lol what for

It's 12181 hits and the number behind the plus sign are robots.txt hits. See the footnote at the bottom of your screenshot.

1. Get a blocklist
2. Enable rate limits
3. Get a proper robots.txt
4. ~~Profit~~ Silence

Check out Anubis. If you have a reverse proxy it is very easy to add, and for the bots stopped spamming after I added it to mine

What is the blog about? It may be increased interest as search providers use them for normal searches now.. or it could be a couple of already sentient doombots.

Please don't be a blog about von Neumann probes. Please don't be a blog about von Neumann probes. Please don't be a blog about von Neumann probes..

Hydrogen bomb vs coughing baby type shit