this post was submitted on 27 Aug 2023
70 points (100.0% liked)

Free and Open Source Software

17932 readers
57 users here now

If it's free and open source and it's also software, it can be discussed here. Subcommunity of Technology.


This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.

founded 2 years ago
MODERATORS
top 2 comments
sorted by: hot top controversial new old
[โ€“] Destragras@kbin.social 9 points 1 year ago* (last edited 1 year ago)

I tried finding information on what indexer they are using. Are they using their own?

Edit: says this in the readme:

The commoncrawl organization for crawling the web and making the dataset readily available. Even though we have our own crawler now, commoncrawl has been a huge help in the early stages of development.

[โ€“] panmeek@lemmy.sdf.org 6 points 1 year ago

this looks VERY interesting!