Correct. All also includes communities fetched but not subscribed to, however these are more like stubs. They are in your database but not being updated with activity since no one is subscribed. At least that’s my understanding.
hawkwind
joined 1 year ago
You should be able to rerun it anytime. It only gets stuff that doesn’t exist on your instance. That’s how it was designed. It is dependent on browse.feddit.de however. :(
I think you're right. People will gravitate to the most stable large instances because their "All" will be as close to 100% as possible without doing anything special. I wrote a script to seed instances and update subscriptions, but it uses a single account that is subscribed to everything so that other users can see everything. That's not something that would normally happen. Maybe that needs to be part of the base software?
Media takes up space. The text from posts and comments is trivial. The database for lemmy.world is only 25 GB. Wikipedia text is only 21 GB.
That is exactly what that means and it's frustrating to say the least, because it's not clear that's what's happening.
It works a lot like like email between instances. Let’s call your self hosted instance “A” and the popular remote instance “B.”
User on A searches for “poodles” and finds a community !poodles@B. When they click the search results: A sends B mail saying “send me the last 10 posts for poodles.” B sends A mail with the posts and the user sees the posts, but none have comments.
If nothing else happens then those 10 posts will just hang out doing nothing on A, but if the user clicks subscribe then A sends another mail to B saying “my user wants to follow poodles.” B replies saying “cool, I’ll send you everything from poodles now.” Now, anything a post or comment happens B checks lots list of subscribing instances and sends copies of them.
If user on A comments on !poodles@B or posts, it creates it on A but sends a mail to B saying “here is some new stuff for poodles!”
When you create that instance, do you immediately need to download and store all the data that has ever been posted to all federated Lemmy instances?
Run my own instance. @Candelestine@lemmy.world is right but there are more details. Federation is not a "sync." When your instance needs to fetch from another instance it will, but it does not get history. You can get a specific comment or post from any time however.
Or perhaps you only need to download and store everything that is posted to the federated Lemmy instances from that point forward?
This is not by default either. Only communities that your users subscribe to will be updated by their "origin" instances.
Or better yet, do you only store what the users on that instance do (i.e. their posts, and posts to the communities hosted on that instance)?
This does happen, but it also stores what your users do on remote instances as well as "copies" of what they interact with. Images (currently the only media hosted by lemmy servers) are linked to thier "origin" as well. So you are storing text of posts and comments.
We should charge for API use now. :)
They want to start an instance for star wars fans because they like captain picard. :)
Commendable that you want to help, but the software needs to handle dead installations as there is no possible way to enforce people to decommission. :) Might be worth dropping just to help the use case of how this gets handled. I feel like you did everything you could.