Data Hoarder

1 readers

1 users here now

We are digital librarians. Among us are represented the various reasons to keep data -- legal requirements, competitive requirements, uncertainty of permanence of cloud services, distaste for transmitting your data externally (e.g. government or corporate espionage), cultural and familial archivists, internet collapse preppers, and people who do it themselves so they're sure it's done right. Everyone has their reasons for curating the data they have decided to keep (either forever or For A Damn Long Time (tm) ). Along the way we have sought out like-minded individuals to exchange strategies, war stories, and cautionary tales of failures.

founded 1 year ago

MODERATORS

communick@selfhosted.forum

Scrapping a reddit post with all its comments? (alien.top)

submitted 11 months ago by Flutter_ExoPlanet@alien.top to c/datahoarder@selfhosted.forum

9 comments fedilink hide all child comments

Hello,

I am not asking about scrapping a whole sunreddit or scrapping all reddit, now just a few select posts,

I noticed that lot of comments get "closed" and you need to open them up by clikcing the (+) button, that can take some time if the post has lot of comments,

I am searching for a way to download the content of one single post, BUT it must handle the (+) itself and it must be able to retrieve all comments (and keeping the parent-son or levels hierarchy).

How can I do that?

top 9 comments

sorted by: hot top controversial new old

[–] steviefaux@alien.top 2 points 11 months ago (1 children)

Nice timing. Going on a cruise over all of Christmas and won't have Internet but taking my laptop with me. Could use this on some of the posts I've saved thst talk about useful powershell scripts or python stuff.

[–] Flutter_ExoPlanet@alien.top 1 points 11 months ago (1 children)

Hope someone have a solution for us:)

[–] PristinePineapple13@alien.top 1 points 11 months ago (1 children)

this github should have what you need

https://github.com/aliparlakci/bulk-downloader-for-reddit

might be hard to get just a few posts, but you can make it do all saved posts

[–] Flutter_ExoPlanet@alien.top 1 points 11 months ago (1 children)

Thanks

[–] PristinePineapple13@alien.top 1 points 11 months ago (1 children)

dm if you have questions, i might be able to help more. been playing around with BDFR for a bit.

[–] Flutter_ExoPlanet@alien.top 1 points 11 months ago

Awesome!

Would you be available in 2 weeks?

I will meddle in this for some project but I have matters to handle before.

[–] southsamurai@sh.itjust.works 1 points 11 months ago

Just a heads up, it's scraping. Scrapping is (simple version) throwing something away or getting rid of it.

It wouldn't matter, but the title is what draws most users to a post, and you might get useless answers because of that, if people don't read the text.

[–] throwaway_bluehair@alien.top 1 points 11 months ago (1 children)

You can still use the Reddit API. https://www.reddit.com/dev/api

If you have a bit of coding know-how this should be rather simple

[–] Flutter_ExoPlanet@alien.top 1 points 11 months ago

Thanks