sunaurus

joined 2 years ago
[–] sunaurus@lemm.ee 5 points 1 month ago (2 children)

We'll get some more control over images when 1.0 comes, I will take a look at it again then, but in general the new user cooldown for image uploads has really significantly reduced the amount of image related abuse we have seen.

@Character_Locked@lemm.ee please understand, nobody on our admin team is getting paid to moderate disgusting images, they are volunteering to do it, and it takes a mental toll. Also, image hosting itself is not free. Our donations right now are barely breaking even with server costs. So realistically, our options are not "current situation vs fully unrestricted image uploads", they are "current situation vs completely burned out admin team and costs increasing beyond donations".

1
submitted 2 months ago* (last edited 2 months ago) by sunaurus@lemm.ee to c/meta@lemm.ee
 

Hey folks!

I'm writing this because funding for the Lemmy project has dropped to critical levels, which could seriously impact its future development.

Thanks to the generous support of our lemm.ee community, our server infrastructure costs are covered, and we even have a few months of runway. I'm deeply grateful to everyone who has contributed - lemm.ee wouldn't exist without your help.

However, infrastructure alone isn’t enough. Our servers run Lemmy software, and without ongoing development, the platform cannot grow or even be maintained.

Lemmy is an open-source project with many contributors, but the vast majority of development work has been carried out by a small group of core maintainers. A few maintainers work full-time on the project, relying solely on donations and occasional grants to support themselves.

I've seen Lemmy development up close, and the maintainers have consistently gone above and beyond what I consider the standard for small open-source teams - they are constantly writing code, mentoring contributors, and keeping everything running. Their work is essential, and without continued support, it cannot be sustained.

If you value Lemmy, please consider supporting its maintainers directly. Every bit helps.

Please check out this post for more details about how to support the maintainers: https://lemm.ee/post/63034576

Thank you for reading, I hope you have a great weekend!

1
submitted 3 months ago* (last edited 3 months ago) by sunaurus@lemm.ee to c/meta@lemm.ee
 

Hey folks!

For the past few hours, lemm.ee has been bombarded with abnormal (almost definitely automated) traffic from a range of different IP addresses. This managed to overwhelm our servers, and we were offline for the past hour or so.

I was in the middle of celebrating my birthday, so response was a bit slow, but I believe we are recovering now, with mitigations in place to try and prevent further issues. Some of you may be inconvenienced by some bot checks when you browse lemm.ee, I am sorry about that, but it's necessary for now.

Sorry for the issues and I hope you have a nice weekend ahead!

1
submitted 4 months ago* (last edited 4 months ago) by sunaurus@lemm.ee to c/meta@lemm.ee
 

Hi folks!

Over the past few months, we have started seeing a significant amount of new user sign-ups. I would like to take this opportunity to welcome all of our new members, and to share some useful resources and info about lemm.ee.

First, some stats

Here is a bar chart of daily new users (this is only counting users which have been approved by our admins):

As you can see from the chart, for most of 2024, we were accepting roughly around 10-20 new users every day. Then, from the start of this year, the daily numbers have been constantly growing. Yesterday, we approved a massive 609 new users on lemm.ee.

The increase in sign-ups is significant enough that I have been taking several steps to improve our monitoring & anti-bot measures, but so far, it seems the vast majority of the new users are completely legitimate real humans! (Thank you all for not being bots 😅)

About lemm.ee

This Lemmy instance is turning 2 years old very soon. It was initially created around the time of the Reddit API changes, when existing Lemmy servers were getting overloaded with new users - lemm.ee was intended to help spread the load. We're now the second largest Lemmy server when it comes to monthly active users.

Our core philosophy for this instance has always been to treat it as a generic gateway to the Lemmy network. I want to provide our users a stable and reliable home for their Lemmy account, so that they can have easy access to all of their communities, regardless of what instance the community is actually hosted on.

We run on some decently beefy hardware, and our setup is fairly customized in several ways in order to ensure a smooth experience for our users (most of the time, this has worked out quite well!). Our servers are currently hosted in Finland.

Our infrastructure has been funded by the community almost from the start through GitHub sponsorships and Ko-Fi donations. I am sure I speak on behalf all of our users when I say that I am extremely grateful to all supporters - you are really responsible for the continued existence of this instance!

Lemmy itself is open source software, and while it has improved massively during the time I have been using it, it definitely still has some rough edges. Please be patient when using Lemmy, and remember that it is being built collaboratively by humans (not corporations), without any intent of ever turning it into a business.

Useful resources

Don't forget to participate!

Communities on Lemmy only work if people actively use them. Even upvoting/downvoting based on quality of content is a great start, but I would really like to encourage you all to comment and even write posts, because that's really the best way to build communities.

If you have any questions or thoughts about lemm.ee or Lemmy in general, feel free to post a comment below this post, and myself or one of our veteran users will definitely respond.

I hope you enjoy your time on lemm.ee, and I wish you all a great week!

 

Hey folks

Just a quick heads up, we will be performing some database maintenance today. Expected downtime is ~15 minutes.

Sorry for the inconvenience!


Update: maintenance complete!

[–] sunaurus@lemm.ee 11 points 4 months ago

Our servers are currently in Finland, but this is subject to change if necessary for financial/technical reasons. We've also used German servers in the past for example. In general we're definitely staying in the EU, though.

1
submitted 7 months ago* (last edited 7 months ago) by sunaurus@lemm.ee to c/meta@lemm.ee
 

Hey Folks

Just a quick note to let you all know about some changes in the lemm.ee admin team. After discussing things with the other admins, we've decided to shuffle around our roles a bit.

Up until now, I’ve been the head admin at lemm.ee - handling infrastructure, maintaining rules and policies, and acting as the main contact person for the admin team.

However, I’ve come to realize that this role has taken a toll on me. While I still love the idea of Lemmy and everything it stands for, being an admin has slowly drained the joy I once had for the platform. The occasional negative experiences have been increasingly difficult for me to shake off. For the past several months, I’ve found myself hesitating to check my DMs or the moderation queue, simply because I’m bracing for some new drama that I no longer have the energy to manage.

After some conversations with the team, we’ve agreed on a plan to ensure my burnout doesn’t negatively impact the instance:
  1. I am stepping down as head admin of lemm.ee.
  2. The new main contact person for the admin team will be @EllaSpiggins@lemm.ee.
  3. I’ll continue to maintain and update the infrastructure behind the scenes.
  4. The rest of the admin team will now handle all moderation issues, managing our policies, and any general admin communications.

It’s been an honor to serve as your head admin, and I’m incredibly grateful for the amazing people I’ve met here. I’m excited to stay involved in a capacity that works better for me and allows me to enjoy this community once again.

See you around!

[–] sunaurus@lemm.ee 27 points 7 months ago (8 children)

lemm.ee hosts a conservative community, which on occasion features borderline transphobia and homophobia.

Hi, lemm.ee admin here, please be sure to report any cases of community moderators not following lemm.ee instance rules directly to admins. The "no bigotry" rule is a treated very seriously at lemm.ee, and our admins will always handle such cases very harshly. We have in the past had to shut down a few conservative-type communities over such things. Having said that, I am not aware of the current moderators of that community allowing, much less featuring, any kind of bigotry (in fact, a quick look over the mod log shows them actively removing bigotry, usually posted by users from outside lemm.ee).

By the way, I know there are a lot of users on Lemmy who really enjoy starting defederation drama (I've lost count at this point of how many users like you I've seen), but is there really nothing better you could be doing on a Friday evening? Maybe go for a walk, spend time with some close ones, read a book, whatever. You are not gaining absolutely anything from what you are trying to do here.

[–] sunaurus@lemm.ee 7 points 8 months ago (3 children)

Hi, there is no free speech policy on lemm.ee, we have very strict moderation when it comes to our rules. We regularly permaban users for breaking our instance rules. We simply don't use defederation as a moderation tool, preferring other tools like user bans, for reasons outlined here: https://lemm.ee/post/35472386

[–] sunaurus@lemm.ee 83 points 11 months ago (6 children)

Interesting! We've had quite a noticeable spike of sign-ups on lemm.ee as well

[–] sunaurus@lemm.ee 2 points 11 months ago

Hey! I'm not really sure about this at the moment. I can tell you that if the authors (or any legal entity) would contact me about this and ask for links to be removed, then I would comply, rather than try to fight it.

[–] sunaurus@lemm.ee 2 points 1 year ago (1 children)

I think it's not really on your side, most likely either just something wrong on kbin.social itself, OR a side-effect of the measures lemmy.world implemented against kbin.social recently.

[–] sunaurus@lemm.ee 6 points 1 year ago* (last edited 1 year ago) (6 children)

They are basically local-only communities on lemmy.world at this point, unfortunately. There is no federation to any other instance for any lemmy.world user posts on those communities.

1
submitted 1 year ago* (last edited 1 year ago) by sunaurus@lemm.ee to c/meta@lemm.ee
 

Hey folks!

For anybody stumbling on this post from outside lemm.ee: I am the head admin of lemm.ee, a general purpose Lemmy instance, which recently turned 1 year old. I am writing this post to elaborate on how we approach defederation on lemm.ee.

Anybody who has been on Lemmy for a while has most likely seen several public defederation drama posts (most recently regarding lemmy.ml, but there have been many many others previously). As an admin, I have probably seen far more than what is visible publicly, as I regularly receive private messages on the topic, ranging from polite questions about federation, to outright demands that I immediately defederate, and even to threats and personal attacks over the fact that I have not defederated some particular instance. It is definitely a topic that will keep coming up for as long as Lemmy exists, which is why I feel it would be useful to condense my current thoughts about it in a single place.

Note that while I strongly believe everything this post contains, it is definitely a subjective topic, and there is no single right answer here. Other instances have completely different approaches to federation compared to lemm.ee, and that’s of course totally fine. The beauty of Lemmy is that everybody can choose their home instance, and in fact, everybody is free to spin up their own instance and run it however they feel is best. For an absurd example, if you want to create an instance which defederates any instance with an “L” in their name, then nobody can stop you!

Quick intro to the lemm.ee federation policy

Very shortly after creating lemm.ee, I wrote down a federation policy, which basically boils down to “we treat defederation as an absolute last resort, and we do not use it as a generic way to curate content for lemm.ee users”. This policy can always be found in the sidebar of the lemm.ee front page.

In practice, this has meant that we have had extremely few defederations, and that we mostly solve problems with other means. I am very happy with the results, as it means that lemm.ee has become a great entry point into the Lemmy network, with very few artifical limitations on who our users are allowed to interact with.

The benefits of federation

I hope that this part of the post is very uncontroversial, but I firmly believe that federation is the absolute strongest feature of Lemmy. While we all know that the concept of federation can cause confusion for new users, this is usually overcome extremely quickly (for example, using the common e-mail providers analogy to explain Lemmy instances). To me, it’s completely clear that the benefits of federation far outweigh the downsides.

For example, by splitting the Lemmy network between thousands of independent nodes, we ensure that:

  1. Any single entity is not a single point of failure for the whole network. Even if the biggest instance goes down tomorrow, their content will still be accessible through all the other federated instances.
  2. The maximum impact of admins is limited to their own instance. As a lemm.ee admin, I can ban a remote user from posting on lemm.ee, but I can’t completely ban them from the entire network.
  3. Private user data (such as ip addresses, e-mails, etc) are never shared between instances. No single malicious instance can harvest user data for the entire network, and extremely privacy sensitive users can always spin up their own instance if they don’t want to put their trust in any existing admins.

One thing which is probably important to note here is that I tend to view Lemmy instances as infrastructure, rather than as communities. I know that there are alternative approaches, as quite a few large instances are in fact run as mega-communities, but that’s not the approach I take with lemm.ee, because I feel like such an approach encourages centralization and negates some of the benefits of federation (if all communities related to one topic condense on a single instance, then that instance does effectively become a single point of failure for a large number of users).

In general, I feel like it should be a goal to encourage and cultivate decentralizing the network through federation as much as is practical, in order to maximize the above benefits.

The downsides of dedeferation

Conversely, defederation has a lot of downsides.

  1. It obviously negates all the benefits of federation mentioned above. Every time two instances defederate, the Lemmy network becomes less redundant, some communities become a bit more centralized, and the danger of malicious admins for those communities becomes much greater.
  2. There is a lot of collateral damage. The most common reason I have personally seen for defederation demands is related to moderation of either a single user, or a handful of users. For example, a lemm.ee user gets into some heated arguments with people from an instance with hundreds of active users, and then links this heated thread to me as proof that the instance should be immediately defederated. However, in this situation, there are hundreds of other users who were not even involved (or even aware of) the thread in question. By defederating, I would be making a decision to cut off every single lemm.ee user from every single one of those hundreds of innocent remote users.
  3. Ironically, defederation actually makes moderation more difficult. It was recently pointed out to me by a user on another instance that they are afraid they can’t effectively moderate communities on lemm.ee, because their instance has defederated several other instances, which means they would not be able to see posts from those instances on lemm.ee communities.
  4. It is extremely easy for malicious actors to abuse. In the year I’ve been on Lemmy, I have already seen two separate cases of users creating accounts on another instance and posting garbage, and then going back to their home instance and demanding their admins defederate over the content they themselves created. Basically, if an instance is known to use defederation as a tool to punish misbehaving users on other instances, then it’s actually quite easy for users to manipulate the situation to a place where admins have no alternative except to defederate.

It seems to me that a lot of users don’t think of such downsides when demanding defederation, or they just don’t consider them as important enough. In my opinion, these are all significant issues. I do not want to end up in a fragmented Lemmy network, where users are required to have accounts on 5 different instances in order to be able to access all their communities.

What’s the alternative to defederation? Should Lemmy become some kind of unmoderated free speech abolutism platform?

I want to be very clear that I do NOT believe in unmoderated social networks. Communities should always be free to set and enforce rules which foster healthy discussions. On top of that, instances should always be free to set and enforce rules for all of their users and communities.

In the case of lemm.ee, we have some instance-wide rules, and we will enforce them on all lemm.ee users, as well as all remote users participating in communities hosted on lemm.ee. For example, we never want to offer a platform for bigotry, so we regularly issue permanent bans for users who want to abuse lemm.ee to spread such hate. In practice, site bans have been extremely effective at getting rid of awful users, whether they are remote or local.

On top of site bans, Lemmy admins also have the option of removing entire remote communities. There are certainly cases where a community might be allowed on instance A, but not instance B - rather than defederating (and potentially cutting off a lot of innocent unrelated users), instance A can just “defederate” a single community.

Finally, a lot of issues can be solved through simple communication between instance admins. Often having a discussion with another admin results in pretty clear alignment over whether some user is problematic, and the user will end up being banned on their home instance.

Being one of the most openly federated large instances with such an approach, we have discovered several things:

  1. If we were to defederate over every rule breaking user or community on the Lemmy network, we would not be federated with any of the large instances at this point
  2. In the vast majority of cases, remote users who have broken lemm.ee rules have ended up banned on their home instance anyway - there is very little additional moderation workload for our admins from being widely federated
  3. If a user truly wants to spread some kind of hate, defederation wouldn’t stop them anyway, as they will just create accounts on any instance which they want to “attack”

The longer I run lemm.ee, the more sure I become that in the vast majority of cases of abusive users, the best approach is to simply hand out site bans.

When is defederation the only option?

Having said all of the above, I still believe that there a few cases when defederation is the best option:

  1. When an instance is abusing the Lemmy network - generating spam, advertising, illegal content, etc - either deliberately, or through inactive admins (this has been the most common reason for lemm.ee to defederate any instance in the past)
  2. When an instance is just causing too much moderation workload. So far, we haven’t experienced this yet on lemm.ee, but I can’t rule out that it could happen in the future.

Conclusion

I hope this post helps clarify my stance on defederation. Like I said in the beginning, I realize a lot of this is subjective, and there are no right or wrong answers - this is just the way we have been (and will be) doing things on lemm.ee. I intend to save this post and link it in the future when people bring up defederation requests. If you feel like I didn’t address something important, please feel free to raise it in the comments!

[–] sunaurus@lemm.ee 17 points 1 year ago* (last edited 1 year ago)

I think there are two separate things I want to address here:

First, agile isn't a project management methodology, it's just a set of 4 abstract priorities and 12 abstract principles. It's very short, you can check it out here:

https://agilemanifesto.org/

Nothing here says that you're not allowed to write documentation, write down requirements, etc. In fact, the principles encourage you yourself as a software team to create the exact processes and documentation that you need in order to meet your goals.

"Working software over comprehensive documentation" does not mean you aren't allowed to have documentation, it just means that you should only write documentation if it helps you build working software, rather than writing documentation for the sake of bureaucracy.

"Individuals and interactions over processes and tools" does not mean that you should have no processes, it just means that the individuals in your team should be empowered to collaboratively create whatever processes you need to deliver good software.

Secondly, in terms of practical advice:

  1. Talk about this problem with your team. Is it hard for others to figure out where requirements came from? Maybe they already have a good method and can share it with you. If it's hard for everybody, then propose improvements to your process, for example, propose some type of design document process as part of building any new features
  2. There are no perfect answers to the question of "how do I safely make non-trivial changes to systems", but the general approach is to ensure that:

a. You have metrics about how your system is used.

b. You have automated tests covering any requirements, so that you can feel confident when making changes to one part of the system that it isn't violating any unrelated requirements.

c. You actually document any confusing parts in the code itself using comments. The most important thing to cover in comments is "why is this logic necessary?" - whenever something is confusing, you need to answer this question with a comment. Otherwise, the system becomes very annoying to change later on.

If you are missing any of the above, then propose to your team that you start doing it ASAP

  1. At the end of the day, somebody is responsible for making product decisions. Is it your team? Or maybe some separate product owner? Sometimes, you just need to communicate with whoever is responsible to figure out if any requirements are still relevant, or if they are now safe to change.
[–] sunaurus@lemm.ee 7 points 1 year ago (1 children)

It's not really a bug, it's just a case where app developers need to update their code to support a small change in the Lemmy API. More details here: https://lemm.ee/post/34259050/12479585

[–] sunaurus@lemm.ee 5 points 1 year ago

Regarding your question:

Lemmy federation basically works by copying stuff from their source instance to all other federated instances. So if I write a comment on lemm.ee, other federated instances will get their own copy of my comment. They will also all know that the "authority" for this comment is lemm.ee.

If an admin on another instance decides to delete their local copy of my comment on lemm.ee, then they are always free to do so (for example, some instances might want to moderate more strictly), but any actions they take like this are limited to their own instance - for the rest of Lemmy, lemm.ee remains the authority for this comment, so individual remote instance admins taking actions won't have any effect on any other instances.

As for the original topic of modlog federation, basically it just boils down to this: just like with the comment example above, Lemmy instances also save a local copy of incoming federated mod logs. The Lemmy software does not yet have 100% coverage in terms of federating mod logs (for example, there are no federated logs yet for instance admins banning remote users), but this coverage has been increasing, and I expect this will eventually get to 100% (just needs more dev time really).

Also, if some instance admins try to tamper with their mod logs, then other instances can still see the real history, because there is no way for an instance admin to delete copies of their mod log from other instances.

[–] sunaurus@lemm.ee 5 points 1 year ago

Banning a local user from a local community does actually federate already

 

Hey all!

Upcoming lemm.ee cakeday

Can you believe that lemm.ee is almost 1 year old? In just a couple of weeks (specifically, on the 9th of June), we will be able to celebrate our first instance cakeday.

I am thinking of compiling some stats about how lemm.ee has been used in its first year, if you have any specific stats in particular you would like to see, feel free to comment below. I will try to accommodate any ideas as I start gathering this info!

Infrastructure updates

A few weeks ago, I posted about plans to make some changes to our infrastructure in order to deal with different intermittent networking issues.. It took a bit longer than I hoped (just did not manage to get enough free time between then and now), but I am happy to report that this work has now been completed! Additionally, I have decommissioned our stand-alone pict-rs server.

With the two changes mentioned above, I believe lemm.ee should now be much more resilient going forwad, and I expect a significantly lower rate of infrastructure-related issues for the rest of the year!

I'll leave a tehcnical overview about the problem & solution below for those interested, but if these details don't interest you, then you can safely skip the rest of this post.


For context, lemm.ee has been hosted on Hetzner servers for most of this year (having migrated from DigitalOcean initially), with everything except our database being hosted on the Hetzner Cloud side, and the database itself living on a powerful dedicated Hetzner server. This mix allows a great amount of flexibility for redeploying and horizontally scaling our application servers, while still allowing a really cost-effective way of hosting a quite resource-hungry database.

In order to facilitate networking between the cloud servers and the dedicated database server (which live in different networks), Hetzner provides a service named "vSwitch". This service basically allows you to connect different servers together in a private network. Unfortunately, I discovered quite quickly that this service is very unreliable. During the short few months that we have been using the vSwitch, we have gone through one extended period of downtime (where the service was just completely broken for several hours), as well as dozens (if not hundreds at this point) intermittent disconnects, where servers randomly lose their connections over the vSwitch. After such a disconnect, the connection never recovers without manual intervetion.

For most lemm.ee users, the majority of these vSwitch issues have been mostly invisible, as we have redundancy in our servers - if one server loses its connection to the database, other servers will take over the load. Additionally, I have generally been able to respond quite quickly to issues by redeploying the broken servers (or deploying other temporary workarounds). However, in addition to a huge amount of these issues which lemm.ee users hopefully haven't ever noticed, there have also been a few short periods of downtime this year so far, as well as a few cases of federation delays. These more extreme cases were generally caused by multiple servers losing their vSwitch connections at the same time.

After several attempts to work around these issues, I decided that we need to migrate away from vSwitch.

As of earlier today, lemm.ee is no longer using Hetzner's vSwitch at all!

I finally found enough time earlier today to focus on this migration, and I was able to successfully complete it. None of our networking is relying on the vSwitch anymore.

In the end, I went with quite a simple solution - I configured a host-level firewall (nftables) on our database dedicated server, which will deny all connections by default. Whenever any cloud servers are added/removed, their corresponding public IP addresses are added/removed in the allowlist of our database firewall. It would have been ideal to do this whole logic in Hetzner's own firewall, but that one unfortunately has a limit of only 10 rules per server, which is just not enough for our setup.

Bonus: our pict-rs server has been decommissioned!

Pict-rs is the software which Lemmy uses for everything related to media (image storage mostly). Initially, pict-rs required a local filesystem to store both files as well as metadata about files. Since the beginning, lemm.ee has used a dedicated server just for pict-rs, in order to ensure we could easily redeploy the rest of our servers without losing any images.

Over the past year, pict-rs has gained the ability to store files in object storage, and metadata in a PostgreSQL database. This meant that the server running pict-rs itself no longer contained any of the important data, so it became possible to redeploy without losing any images. Additionally, this meant that it would be possible to run multiple pict-rs servers in parallel.

While we had already migrated our pict-rs server to use object storage and PostgreSQL several months ago, we still had the single dedicated pict-rs server up until today. I have been planning for a while to decommission this server, and start running pict-rs directly on each one of our Lemmy application servers. Earlier today, I was able to complete this plan. This should hopefully mean that our pict-rs server is less likely to get overloaded, and it also means a tiny reduction in our overall monthly infrastructure bill (due to one less server running).

With the above changes, I think our infrastructure has become more robust, and hopefully, we will experience less issues with images, federation, and general downtime going forward.


That's all from me for now. Feel free to leave any thoughts or questions in the comments, and as always, I hope you're having a great day!

 

Hey folks!

This is a quick notice about a change to our moderation policy.

We have had a policy on lemm.ee for administration and federation nearly since the very beginning. This policy has also always included a section about moderator responsibilities. Today, we have made two changes to this policy:

  1. The policy has been renamed to Policy for administration, moderation, federation - this is to make it clear that the policy is also relevant for mods
  2. We have introduced a new responsibility for moderators, they must "Ensure that they only provide accurate and clear reasons for mod actions".

The reason for the addition is that mod log actions federate out to other instances, and are more or less permanent (due to how Lemmy and federation works right now). This means that users do not really currently have any easy way to clarify or defend themselves against inaccurate accusations in the mod log.

As always, I am very grateful to all mods for your efforts in building awesome communities on lemm.ee. I hope you can understand why this new policy is necessary - I do not want to make your lives more difficult, the goal is to just try and reduce any mod log related misunderstandings in the future.

Thank you for reading and have a nice day!

1
submitted 1 year ago* (last edited 1 year ago) by sunaurus@lemm.ee to c/meta@lemm.ee
 

Hey folks!

We unfortunately had about half an hour of unplanned downtime today. This was caused by an issue with our hosting provider. The issue is solved for now, and I am planning to make some changes to prevent similar issues in the future. Sorry for the inconvenience!


Technical details

Our servers are communicating with our database over Hetzner's "vSwitch" service. Unfortunately, this service seems to be quite flaky - over the past few months, I have had to deal with the connection just dropping without recovering many times. Mostly this has not resulted in any noticeable downtime, as we have redundant servers, so even if one of them stops working, it won't affect lemm.ee users. However, in this instance, all of our API servers lost their connection to our database at the same time, which resulted in actual downtime.

I have now decided to migrate our setup away from the vSwitch in the near future to hopefully stop these issues for good. Should be possible to do this migration without any downtime, I just need to set aside some time to actually create an alternative solution for us, most likely over the coming weekend. I will update this post once the migration is complete.

Update: the migration is now complete! You can read more here.

 

Hey folks

This is just a quick heads up that I need to perform some maintenance & upgrades on our database server, which unfortunately will require downtime. I don't expect the downtime to last for longer than 2-3 minutes, but just wanted to give a heads up first so you know not to be concerned.

That's all, hope you have a great week!

Edit: maintenance complete!

 

Hello, friends!

TL;DR: I am working on a new Lemmy frontend in nextJS. There is still much work to be done, but you can already have an early look at https://next.lemm.ee

First of all, quick note to lemm.ee users: I am making this announcement post in !meta@lemm.ee, as this is also a notice that I will be hosting an alternative frontend (lemmy-ui-next) for the first time on lemm.ee. Going forward, I will post updates about lemmy-ui-next in a separate dedicated community: !lemmy_ui_next@lemm.ee. If you're interested in future updates, please subscribe there!

What is lemmy-ui-next?

Lemmy is generally accessed through some kind of frontend UI. By default, Lemmy provides its own web interface (lemmy-ui), which you can find on the front page of most Lemmy instances (including lemm.ee). There are also several other independent frontends, for both the web and different mobile platforms, which I'm sure many of you are familiar with.

Lemmy-ui-next is a brand new alternative frontend, built from the ground up with modern and popular tooling - a framework known as NextJS. Lemmy-ui-next has (or aims to have) the following high-level features:

  • Open source (AGPL)
  • Drop-in replacement for lemmy-ui - same exact URL structure, so all existing links will continue working
  • Very plain & minimalistic UI, strongly inspired by other link aggregator sites (of course including the original lemmy-ui!)
  • Very basic and "typical" NextJS architecture, to encourage open source contributions
  • Fully functional even when JavaScript is disabled (but works better with JS enabled!)
  • Optimized data transfer between your browser and the server (filtering out only relevant data from the Lemmy API, caching, memoization)
  • Strong focus on privacy and security (all authentication with the Lemmy API is done through secure httpOnly cookies, user IP addresses are not leaked to external image hosts, etc)

What is the current status of lemmy-ui-next?

I have mentally split the initial work I want to complete into 3 milestones:

  1. Lurk - All read-only features of Lemmy
  2. Participate - Voting/posting/commenting/DMs/reports, etc
  3. Moderate - Handling reports, creating & managing communities, etc

I am now nearing completion of the first milestone. It's not 100% there yet, but you can already log in, browse, subscribe to communities and even vote. Some things may still look a bit wonky, and some features are still missing, but the core experience is getting there.

In terms of code contributions, I would ask anybody who is interested in getting involved to contact me first before working on anything. I am not looking for PRs just yet - the code structure is still a bit loose, and I am redefining it as I add more stuff. I would ideally really like to complete the first 3 milestones before opening things up for external contributors.

Who can use lemmy-ui-next?

At the moment, it is only hosted on this instance, at https://next.lemm.ee. I do not yet have any formal instructions for running it on other instances, but generally speaking, it is a simple NextJS app - to deploy it, you just need to do: npm install, npm run build and LEMMY_BACKEND=https://<your lemmy api here> npm run start.

Why not just improve lemmy-ui instead?

Lemmy-ui is an extremely important and valuable project. There has been a significant amount of hard effort put into it so far, and nobody can refute that it is the frontend which has really carried Lemmy to this point.

Unfortunately, there are some architectural problems with lemmy-ui (mostly related to how data is fetched and how sessions are stored in memory), all of which would require quite a significant rewrite to fix. Additionally, I think that the core technical solution used for lemmy-ui is just a bit too obscure, which has been an obstacle to my own contributions, as well as to contributions by others. If a rewrite is on the table anyway, then I believe a different technology is the best way forward.

Why not work on lemmy-ui-leptos instead?

Lemmy-ui-leptos is another rewrite of lemmy-ui, which is being lead by Lemmy maintainers. It is based around a Rust web framework called Leptos. I think this is really cool tech, and will be happy to host lemmy-ui-leptos on lemm.ee in the future as well.

There are a two key reasons why I personally decided to start working on another alternative, though:

  • I have heard from several people on Lemmy that they feel like Leptos is a big barrier to entry in terms of them contributing
  • Even for myself personally, I am very comfortable (and think I can move very fast) when working on something like NextJS, but with Leptos, I think the learning curve would be quite big and I would get much less done with any time I invest into it

My hope is that by providing a very vanilla alternative, I can provide an outlet for potential open source contributors who would like to work on Lemmy, but aren't prepared to do it with Leptos.

Does this mean that lemm.ee will change in ways I don't like?

First, let me be clear: lemm.ee will always host the default Lemmy frontend. This means lemmy-ui for now, and most likely lemmy-ui-leptos in the future.

I am however considering the possibility of switching things around at some point in the future, so that lemmy-ui-next will be hosted directly on lemm.ee, and lemmy-ui will be accessible on a different subdomain (like ui.lemm.ee). This would only happen once I have completed all 3 milestones for lemmy-ui-next. The main reason I am considering this is that I feel like I will always be in the best position to offer technical support to users on the frontend which I am myself maintaining. If you have any thoughts about this potential change, please let me know in the comments below!

That's about it for now!

This is something I've been thinking of doing for a while now, and I'm very excited to finally get the ball rolling! If you have a chance, please feel free to check out what https://next.lemm.ee looks like so far, and please let me know if you have any thoughts or feedback!

 

Hey

This is just a quick heads up that our host, Hetzner, has been experiencing networking issues today, which has caused some downtime for lemm.ee.

I have a workaround in place for now, so we should (fingers crossed) be recovering at the moment, but I am still waiting on the proper solution from Hetzner. You can track their issue here: https://status.hetzner.com/incident/9406c500-9c8b-48be-9591-a73691134096

Also, this is a good opportunity to remind everybody about https://status.lemm.ee - you can be sure that I will provide updates on that page as soon as I am aware of & dealing with any issues. I have been posting status updates for the current issue there as well.

Sorry for the inconvenience and I hope you have an otherwise great day!

UPDATE: Hetzner claims they have fixed the issue, but the problems have not been resolved for lemm.ee servers yet, so I am keeping my temporary workaround active for now. Will continue troubleshooting this tomorrow.

UPDATE 2: Hetzner has now fixed their issue, and our network has been restored to its original optimized state.

view more: next ›