Selfhosted

59955 readers

461 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam.
Posts here are to be centered around self-hosting. Please ensure it is clear in your post how it relates to self-hosting.
Don't duplicate the full text of your blog or git here. Just post the link for folks to click.
Submission headline should match the article title.
No trolling.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago

MODERATORS

curbstickle@anarchist.nexus

curbstickle_lw@lemmy.world

Looking for some selfhosted webscanner/scraper (lemm.ee)

submitted 2 years ago by yournamehere@lemm.ee to c/selfhosted@lemmy.world

7 comments fedilink hide all child comments

hi, i'm kinda out of ideas -maybe you know some selfhostable solution.

I want to monitor some of my pages for gdpr compliance and have reports on that.

So far i tried selenium container with k6, i tried selenium locally with some Python automation and even gave phantomJS with webXray a try. In any case sth. is missing...like selenium wont grab the thirdparty cookies, webxray code has outdated Python and so on.

You guys have any suggestion on how i could selfhost my own automated toolset to check entire webpages for all cookies and thirdpartyscripts? Ppl make changes which might have impact on the gdpr compliance and i dont want to check manually. How would you do it selfhosted?

top 7 comments

sorted by: hot top controversial new old

[–] morras@links.hackliberty.org 5 points 2 years ago (1 children)

The EDPS open-sourced a cookie inspection tool:

https://edps.europa.eu/edps-inspection-software_en

[–] yournamehere@lemm.ee 1 points 2 years ago* (last edited 2 years ago) (1 children)

oh wow...how did Google not suggest this to me? thanks mate.

edit: got i running...works like a charm. just great. thanks so much.

[–] morras@links.hackliberty.org 2 points 2 years ago

You are welcome :)

[–] steersman2484@sh.itjust.works 3 points 2 years ago (1 children)

You could try Puppeteer, but I don't know if you can check third party cookies.

Also I can recommend using Apache Airflow for automation. It is like a fancy crontab with a webui and email notifications, and lots of other stuff.

[–] yournamehere@lemm.ee 2 points 2 years ago

thanks. was thinking about airflow or n8n for automation but i still cant find a way to check the thirdpartycookies. someone wrote firefoxwebdriver could do it but you'd need to extract from the profile an stuff...nothing really worked.

[–] 0xD@infosec.pub 2 points 2 years ago (1 children)

Check out nuclei or maybe greenbone.

[–] yournamehere@lemm.ee 2 points 2 years ago

those look exciting! thank you very much.