While I was asleep, apparently the site was hacked. Luckily, (big) part of the lemmy.world team is in US, and some early birds in EU also helped mitigate this.
As I am told, this was the issue:
- There is an vulnerability which was exploited
- Several people had their JWT cookies leaked, including at least one admin
- Attackers started changing site settings and posting fake announcements etc
Our mitigations:
- We removed the vulnerability
- Deleted all comments and private messages that contained the exploit
- Rotated JWT secret which invalidated all existing cookies
The vulnerability will be fixed by the Lemmy devs.
Details of the vulnerability are here
Many thanks for all that helped, and sorry for any inconvenience caused!
Update While we believe the admins accounts were what they were after, it could be that other users accounts were compromised. Your cookie could have been ‘stolen’ and the hacker could have had access to your account, creating posts and comments under your name, and accessing/changing your settings (which shows your e-mail).
For this, you would have had to be using lemmy.world at that time, and load a page that had the vulnerability in it.
Good point, I’ll update the post.
Also I am curious, what’s the easiest way to currently reach the admins in case this happens again somehow? Two of them on their account have been seemingly inactive for a month and as per your own statement you rarely check your notifications and dms. Is there a discord somewhere for it?
Mail: [email protected] Mastodon: @[email protected] Matrix: https://matrix.to/#/#lemmy-support-general:discuss.online
Why wasn’t there an info on /lemmy-world.statuspage.io ?
I think the admins that were on it didn’t think of updating the status page…
Would it be a good idea to have a secondary email not attached to lemmy.world in case of a domain hack?
[email protected]
The mail server records of a domain name do not usually point to the same server as other services like Lemmy.
Domain registrar hack could happen too
deleted by creator
So all our cookies are negated now with the JWT changed, and we just needed to login again? Can attackers have stolen our cookies in order to use our accounts to post as if it was us? I’m sure they were only interested in admin cookies, so most others were “useless” to them? I see nothing wrong with my posts so I should be safe, right?
Prior to the JWT secret being rotated, yes, they could have authenticated as you. The tokens are now all invalid and useless
Probably. I had to re-login myself.
If you think they could change your password:
YES, they could.
They could have changed the email => “Forgot PW” and with that you lost ur account.
I think I’ve lost my account, I clicked Forgot Password and nothing came into my mailbox. This account is the one I made just now.
My old account:
If you see that account post or comment on anything, please report it
Edit: Nvm, I use another email to sign up for Lemmy and forgot about it
Report it directly to Ruud or otherwise he will just delete it.
actually nevermind, I forgot that I use a different email for Lemmy, I can log back in now
It happens to all of us. Additionally, assuming that you’ve come here recently, there’s not much data on it, and it being deleted will not be that much of a big deal.
deleted by creator
Nice work on the recovery, especially from a 0-day.
deleted by creator
They defaced it with dicks and changed the federation list to be only threads.net. I don’t think it was a state sponsored chinese hacking group. :)
I’m ok with the dicks but the threads are TOO FAR!!! shuffles off to the angry done**
Thank you all for staying on top of it.
right after the update we also had most of the serverlist cleared except threads.net (which was the last one added so i assumed it was some bug) – otherwise nothing appears to be touched on this instance tho.