I am interested in checking out the historical growth of a particular community. Lemmy Explorer crawls for data about the Lemmyverse every 24 hours or something, and that data is made available on their website. But I can only find where to download the latest data. Is there somewhere that I can find historical data? Does Lemmy Explorer archive these anywhere, or does it just overwrite the previous data each time it crawls?

you are viewing a single comment's thread
view the rest of the comments

[–] testAccount@programming.dev 1 points 7 months ago (1 children)

Mostly because Lemmy has a hard time keeping up with the federation

https://github.com/LemmyNet/lemmy/issues/4529

[–] freamon@lemmy.world 1 points 7 months ago* (last edited 7 months ago) (1 children)

Oh, right. The chat on GitHub is over my head, but I would have thought that solving the problem of instances sending every activity 2 or 3 times would help with that, since even rejecting something as a duplicate must eat up some time.

[–] testAccount@programming.dev 1 points 7 months ago

Yes but now it must wait for acknowledgement of a request (activity), before sending the next one. If one request takes 333ms means you can do max 3 requests per second. Now big instances like lemmy.world have activity above that so instances too far will perpetual lag behind