Looks like it works.

Edit still see some performance issues. Needs more troubleshooting

Update: Registrations re-opened We encountered a bug where people could not log in, see https://github.com/LemmyNet/lemmy/issues/3422#issuecomment-1616112264 . As a workaround we opened registrations.

Thanks

First of all, I would like to thank the Lemmy.world team and the 2 admins of other servers @[email protected] and @[email protected] for their help! We did some thorough troubleshooting to get this working!

The upgrade

The upgrade itself isn’t too hard. Create a backup, and then change the image names in the docker-compose.yml and restart.

But, like the first 2 tries, after a few minutes the site started getting slow until it stopped responding. Then the troubleshooting started.

The solutions

What I had noticed previously, is that the lemmy container could reach around 1500% CPU usage, above that the site got slow. Which is weird, because the server has 64 threads, so 6400% should be the max. So we tried what @[email protected] had suggested before: we created extra lemmy containers to spread the load. (And extra lemmy-ui containers). And used nginx to load balance between them.

Et voilà. That seems to work.

Also, as suggested by him, we start the lemmy containers with the scheduler disabled, and have 1 extra lemmy running with the scheduler enabled, unused for other stuff.

There will be room for improvement, and probably new bugs, but we’re very happy lemmy.world is now at 0.18.1-rc. This fixes a lot of bugs.

  • @Dogs_cant_look_up
    link
    1201 year ago

    So no more random sports posts at the top of every thread? That was a weird bug.

    • @figaro
      link
      941 year ago

      You WILL know the score

      • @theragu40
        link
        211 year ago

        Scores will continue until morale improves!

    • @veroxii
      link
      241 year ago

      That was because of websockets. Which are no more in this version so yes all those “random things” bugs from multiple tabs should be fixed.

    • @sk1nnyjeans
      link
      221 year ago

      I was so confused by those and kept backing out my feed and going back to the post, thinking I tapped on the wrong post or maybe the app opened the wrong post. I’d scroll down and find the right comments though, I couldn’t make any sense of it. Glad I wasn’t crazy lol