Data Hoarder

249 readers

3 users here now

We are digital librarians. Among us are represented the various reasons to keep data -- legal requirements, competitive requirements, uncertainty of permanence of cloud services, distaste for transmitting your data externally (e.g. government or corporate espionage), cultural and familial archivists, internet collapse preppers, and people who do it themselves so they're sure it's done right. Everyone has their reasons for curating the data they have decided to keep (either forever or For A Damn Long Time (tm) ). Along the way we have sought out like-minded individuals to exchange strategies, war stories, and cautionary tales of failures.

founded 2 years ago

MODERATORS

communick@selfhosted.forum

travel info wanted (blablacar, flixbus, trains, etc) (fedia.io)

submitted 1 month ago by ciferecaNinjo@fedia.io to c/datahoarder@selfhosted.forum

4 comments fedilink hide all child comments

The websites of blablacar, flixbus, and various trains sites are hostile toward the idea of travelers getting all the data they need to plan a trip.

E.g. Flixbus is tor-hostile, but even when access is granted you cannot just ask for the cheapest trip from A to B over a range of days. The motherfuckers force us to search one day at a time and just one destination at a time.

Fuck that. How can we get the data? I know these sites have bot-hostility so scraping it seems like a huge effort.

Some countries have “open data” laws that requires sharing the data, but that only works if the gov gets the data to begin with. If the gov does not get the Flixbus data, then there is no legal requirement to share it.

top 4 comments

sorted by: hot top controversial new old

[–] rglullis 2 points 1 month ago (1 children)

I think the issue here is that pricing for trips is heavily dependent on the user profiling and segmentation. It will be hard to find "public" datasets because a query about trips from point A or to point B will vary simply by matter of who is asking.

[–] ciferecaNinjo@fedia.io 2 points 1 month ago (1 children)

Personalised pricing is evil indeed. You make an interesting point because exposing that evil likely gives an angle on why carriers resist open data which I had not considered.

When I raised the question, I did not mean to limit the request to official sources. In fact, I somewhat expect that a dataset would come from an independent 3rd party. Even if the prices are biased for a particular person, it’s relative pricing that’s most interesting anyway.

Finding the cheapest is quite useful even if there are slight markups/markdowns with whatever vendor sells the ticket. Flixbus discriminates against Americans by adding $1 to every ticket from US IP addresses, but a US dataset would still help me decide outside of the US which route is the cheapest.

Note as well that routes and schedules are useful even without accurate pricing -- for BlaBlaCar in particular because people offering seats in their car have no periodic schedule.

[–] rglullis 2 points 1 month ago (1 children)

Is there any type of browser extension that could help people crowdsource data? It's the only way I see how to build this dataset.

[–] ciferecaNinjo@fedia.io 2 points 1 month ago

I had the same idea but AFAIK it does not exist.