Hmm, getting origin servers to expose themselves this way is a clever hack. As noted, any bad actors probably already know this trick to bypass Cloudflare/whatever anti-DDOS layer.
As a fix, I guess you can either send your server's outgoing connections through a proxy/VPN or use your hosting company's firewall to block all non-Cloudflare inbound traffic.
Could you share a screenshot of your final result?