this post was submitted on 05 Jul 2023
867 points (99.1% liked)
Lemmy
12524 readers
104 users here now
Everything about Lemmy; bugs, gripes, praises, and advocacy.
For discussion about the lemmy.ml instance, go to !meta@lemmy.ml.
founded 4 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
It's not exactly something that you can force. If X amount of users want to join an instance Y, the instance should be able to provide capabilities to host those users. Besides, horizontal scaling provides other benefits, stability is the main one - if one server instance goes down, others can immediately pick up the slack.
Lemmy.ml did exactly that and is one reason why lemmy.world got to be that big.
Once the server capacity is reached the instance should be closed and people will just go to another one.
I don't understand why people feel so happy about lemmy.world being ahead of the rest. It's against the point of the fediverse and has risks: the instance can be sold, can make decisions to put advertisement etc. It's like people didn't really understand what was wrong with reddit to begin with and how the fediverse tries to be different.
Why? And who pays for that?
Why not? I don't see the drawback to develop ability to do horizontal scaling. If the instance owner doesn't want to add additional servers, it's up to them. Obviously they paid for it if they decide to add.
Just to be clear, horizontal scaling means multiple servers handling same instance, it can be the backend service to allow handling more traffics, or multiple db to reduce database loads.
Additionally it allows high availabilities, so if one of the backend service is down (either unexpectedly or do rolling update) the other service can still active so the instance can still be accessed by users
All of your points only considers the community itself, which is not my argument. And that can already achieved by just doing Vertical scaling like most instance currently do, I think even lemmy.world just using 32 core now (google cloud can possibly run with more than 200 vCPU).
I'm mostly approach this from technical standpoint. It's impossible to have 100% uptime if there's no horizontal scaling capability. For example on updating version, currently the instance will need to shut down for maintenance until it's finished and usually there's still some issue to fix. If horizontal scaling exist, the instance can update server (or add additional one), move the traffic a bit to test it, and then fully rollout if everything going well.
Not to mention a hardware failure, which could take a couple of hours to fix at least, some mental health communities should stay online at all time, someone mentioned there is research showing when a person is suicidal there are a couple of hours he is vulnerable, and there is some research showing online support can improve mental health.
I understand. But the point I am trying to make is that makes no sense to worry about technical issues now. Not only it is a premature optimization, it is the kind of metric that is actually damaging to us.
What do you prefer? A server that can handle hundreds of thousands of users with 5 nines of uptime, focuses on "Web Scale" and ends up replicating all the issues from Facebook/Twitter/Instagram/Reddit, or an instance that is more aligned with the ideas of the SmolWeb and that is more likely to be a net-positive force in your life?
Dude, I think you're just ignorant of how web hosting works. Every single site you visit is hosted on probably dozens or more servers so that it can load balance or guarantee better uptime. It's normal. It's weird to be in a place that is only on one server.
Being able to host a stable site doesn't mean everything is suddenly moving into one instance either. The NBA subreddit for example, a single community, has millions of members. Lemmy can't handle anything like that. And technologically having no way to support large communities is a guaranteed way to kill your app.
You also seem to be very in favor of spreading out and decentralizing.... except for Beehaw. Wonder why you're such a purist for decentralization except in this case. Weird. Being able to defederate, make moderation decisions for yourself, and making big decisions like that to defend your community is the whole point of these sites. Maybe you should go back to Reddit if you aren't able to handle it. And for the record, you'd have to be blind to not see moderation controls are lacking at best for this brand new actively being developed site.
I run a managed hosting service for Mastodon and Lemmy, but yeah...
Hacker News: one single FreeBSD box. Not even a database.
Also, your cargo-cult is showing... talking about "load balance" as a guarantee of uptime is the same as justifying using Mongo because it is webscale
You sound like an old script kiddie who says they're a hacker cause they ran a script from a forum. If it wasn't obvious, I'm talking about actual web architecture. Not hobby junk. Managing to standup a tiny virtual instance for a few people does not mean that you understand anything.
As I said, this I basic architecture shit. Like, an intern would understand the idea kinda basic.
???? Are you unironically implying that a site with a backend that has multiple servers stood up to spread the load won't have tremendously better capacity, redundancy, and as a result better uptime than a single hobby pc in your living room or whatever you have setup?
Can you please stop with the unnecessary snark and this silly attempt at dick-measuring? Are you upset at something?
No. I am saying that the majority of websites out there don't need to pay the costs or worry about this.
Good engineering is about understanding trade-offs. We can be talking all day about the different strategies to have 4, 5 or 6 nines of availability, but all that would be pointless if the conversation is not anchored in how much will be the cost of implementing and operating such a solution.
Lemmy - like all other social media software - does not need that. There is nothing critical about it. No one dies if the server goes offline for a couple of minutes in the month. No business will stop making money if we take the database down to do a migration instead of using blue-green deployments. Even the busiest instances are not seeing enough load to warrant more servers and are able to scale by simply (1) fine-tuning the database (which is the real bottleneck) and (2) launching more processes.
Anyone that is criticizing Lemmy because "it can not scale out" is either talking out of their ass or a bad engineer. Possibly both.