this post was submitted on 21 Jun 2023
28 points (100.0% liked)
Lemmy
12538 readers
1 users here now
Everything about Lemmy; bugs, gripes, praises, and advocacy.
For discussion about the lemmy.ml instance, go to !meta@lemmy.ml.
founded 4 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
It's dumb that someone needs to interact with community in order for it to federate with your instance. Like, how are you supposed to find it in the first place? It makes it difficult for communities to grow on instances that are not mainstream which makes decentralization useless.
I just did some math and assumed there are 700 (some instances are blocking other instances) instances and 12 000 communities. 700*12 000 = 8 400 000, users across the platform need to copy url of community and paste it into search this many times to make the platform fully federate with everything. Numbers were taken from here: https://lemmyverse.net/
this number is only completely relevant for someone on an instance all by themselves or with no communities at all. And discounts instances that are or will de-federate either partially or fully. It also assumes some need to be a part of all 12,000 communities. I think tools like you linked solve this issue anyway. I personally believe to a certain extent every community being federated to every instance kind of defeats the purpose of federalization.
But that's not my point though? The point is to federate with all sublemmies of FEDERATED instances. The problem is that when you federate other instance it doesn't federate sublemmies of that instance automatically which limits interactions between instances by a HUGE amount
It's a risk reward question then. That would 100% slowdown the initial federation if it needed to pull in every community and even if that was accepted, should every instance constantly poll any instance it knows about for new communities? Also you aren't guaranteed to need all those instances anyway and then that's just a waste of space and processing power. Correct, it limits interactions but only to what's necessary which allows instances to be ran on lower powered hardware, allowing more people to join in. With the possibility of third party tools I don't see much of an upside of building that into lemmy.
In order to avoid this restriction you would need a global instance discovery mechanism, which is extremely hard to implement without a central server that keeps a list of all instances in the network. And if you do implement instance discovery through a central server you really are losing the whole point of decentralization.
Additionally, it's good that each instance does not federate with everyone else by default. If it did, it would have to process all activity and keep a local copy of all the content in the entire network. This would be insanely inefficient, and make it prohibitively expensive to run even a tiny instance with 1 user and no communities.
Decentralization isn't useless if you can't immediately see everything in the network, come on... We're just spoiled by centralized services.
I don't think you do. Instances should merely reach out to other instances it's federated with periodically to get a list of communities and some of their metadata. Ideally, they could ask all of those other instances to notify it when a community is added, modified, or deleted, and then store that metadata.
That should be pretty easy to implement, and maybe it already has, idk.
Sure, but this isn't finding new instances, just new communities on known instances. Indeed, this is not difficult to implement. The reason it's not done already is for resource economy. A lot of instances are already struggling to scale, making them process and store a lot more content with little value for most users of the instance isn't feasible for a lot of servers right now.
A list of communities isn't "a lot more content." Just run it once daily and the problem is solved. Instances don't need to store posts, just community names available on that instance.
If the issue is finding new instances, I think it's fine for it to take some time the first time someone tries to find a community on that instance. But after that, it should immediately have a list of communities from that instance.
Ummmm... All you need is some bots on each instance that automatically will interact once with communities known on lemmyverse.net and boom, you have unlocked full federation for every user on those instances. You are not losing any privacy through that, it just skips the steps where user has to manually index a sublemmy before it federates and makes platform more usable.
Sure, but now this system has a dependency on the "centralized" lemmyverse.net service. And also your instance now has to receive and store a copy of almost the entire network's content. Lots of instances are already struggling to sustain the load, this would make the problem even worse.
If a single instance decides that it can sustain the increased load and doesn't mind depending on lemmyverse.net sure, nothing's stopping them. But it shouldn't be the default behavior for all instances.