this post was submitted on 24 Jun 2023
233 points (100.0% liked)
Technology
37739 readers
529 users here now
A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.
Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.
Subcommunities on Beehaw:
This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
It should always be possible to scrape and make a feed. I use this at the moment: https://github.com/trashhalo/reddit-rss
It would be a fun project to learn how to make a web scraper.
that is only half the work/job, other half is learning how to avoid ban.
I've been scraping 16+ years and it is easy till they get onto you and block you.
what do you use the most to scrape ? i'm thinking to learn scrapy.
Scrapy is really simple and straightforward.
You can start really fast.
I scraped literally everything :)
Personal hobbies, business wise,...
Scripts, apps, databases, got data for a lot of stuff
I am now usually working with APIs, but still need scraping sometimes.
Thank you. Would you mind if i send you PM? Need to ask something relating to my job.