Technology

What I was responding to here is the idea that of running an automated program on information shared without permission. In that case, the fair use argument becomes very difficult to make, in my opinion. Search engines and other forms of analysis is definitely allowed, but those copies are provided through legitimate means. Downloading articles from behind a paywall and sharing them isn’t the same as indexing publicly available web pages.

I'm not saying you should get anything through illicit means, you could just view the web page yourself rather than sending it to anyone else. For example LAION, provides links to internet data, they don't download or copy content from sites. By visiting it yourself, you'd dodge all those problems.