I followed some popular communities from a Mastodon server ages ago, and then unsubscribed when I was satisfied that it was working.
However, lemmy seems to have some problem with the way Mastodon sends an ‘Undo/Follow’, so it’s still been sending traffic ever since. Recreating it on a lemmy server I booted, journalctl shows this error:
WARN Error encountered while processing the incoming HTTP request: lemmy_server::root_span_builder: Unknown:
0: lemmy_apub::insert_received_activity
with ap_id=Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("activitypub.academy")), port: None, path: "/e97071d0-54e4-4527-9865-e44cf1a55970", query: None, fragment: None }
at crates/apub/src/lib.rs:191
1: lemmy_apub::activities::following::follow::verify
at crates/apub/src/activities/following/follow.rs:78
2: lemmy_apub::activities::following::undo_follow::verify
at crates/apub/src/activities/following/undo_follow.rs:66
3: lemmy_apub::http::community::community_inbox
at crates/apub/src/http/community.rs:50
4: lemmy_server::root_span_builder::HTTP request
with http.method=POST http.scheme="https" http.host=38ec-77-100-144-83.ngrok-free.app http.target=/c/test1/inbox otel.kind="server" request_id=70c556c1-9f2c-421d-90f3-826e9d12879c http.status_code=400 otel.status_code="OK"
at src/root_span_builder.rs:16
I reconfigured the JSON that Mastodon sends to be more like a Lemmy one, signed the http request with a new set of keys, and sent it via a command-line script. Technically, this works, in that I can use the script to subscribe and unsubscribe from lemmy communities. Unfortunately, it doesn’t work with the private key for the old Mastodon user and lemmy.ml (which rejects the http requests). I don’t know why, but it has stalled my attempts to unsubscribe from my side.
As you’re likely aware, lemmy.ml has bouts of sending out an absolute firehose of info at times, and it’s a waste of your resources sending them to me (who’s resorted to 403ing everything, so they don’t overwhelm the ActivityPub server I’m building).
So, if possible, can someone have look what a user from lemmon.website is still subscribed to (memes and asklemmy, certainly) and unsubscribe them please?
I can provide some credentials on lemmon.website if you require.
Thanks.
If you run the server could you just defederate?
https://fedi.tips/how-to-defederate-fediblock-a-server-on-mastodon/
Reading your link, a defederation is just blocking the traffic from my end, which I’m already doing by having nginx reject everything with a 403 code.
I’m hoping that I can stop the traffic from being sent in the first place.
Are you using anything like cloudflare, something with an external firewall? You could block incoming requests there before it reached your backend?
I’m not no, just the cheapest VPS I could find.
The requests never reach my backend, 'cos they’re blocked by nginx rather than it reverse-proxying them like it does for the traffic I’m interested in. So, thankfully for my poorly-programmed backend, it doesn’t have to process the deluge from lemmy.ml. It’s a bit of blunt instrument though, 'cos it just blocks everything. In the future, there’s likely to be some quieter communities on lemmy.ml I’d like, but right now, I can’t follow them without also getting everything from ‘memes’ and ‘asklemmy’.
I’m also thinking about how lemmy seems to get behind on it’s federation activities, and it would maybe help if they didn’t generate them in the first place for recipients who don’t want them.
I’m essentially asking someone to stop coming to my house and knocking on my door. I could ignore the knocks, or put a sign up, or employ someone to stand in my garden and stop them. These are strategies to cope with the problem, but I think the real solution would be if they just stopped coming.
Unfollow activities from Mastodon should be supported. Can you try to refollow and then again unfollow the affected communities? Also check the server logs at that time. Its also possible that other users from your instance are still following the communities.
I’m not sure they are supported.
Firstly, I should clarify that lemmon.website is mine, and I’m the only user that’s ever been on there.
When I was running Mastodon, I noticed that, after I’d unsubscribed to the lemmy.ml communities, there was still entries for incoming activities in the systemd journal. Before I stopped running Mastodon, I unsubscribed from all lemmy.world communities. After I started up a new, basic server using Rust’s hyper library, I saw that all activity from both lemmy.ml and lemmy.world was still POSTing to /inbox
I think what happens is that, when a Mastodon user unsubscribes, the software longer views the community’s public key as valid, so it doesn’t show the posts, etc, but you haven’t ‘really’ unsubscribed because lemmy has errored. Here’s how I’ve tested this.
-
Start a brand new lemmy instance, on a linux VM in Windows, using ngrok to tunnel though from a ephemeral URL. So, right now, there’s lemmy service you can see running using:
curl --header 'accept: application/activity+json' https://f2d0-77-100-144-83.ngrok-free.app/api/v3/site -
Log in to a Mastodon server that’s designed to let users experiement with ActivityPub (https://activitypub.academy/), to subscribe and unsubscribe to a new-created test community. It looks from there like:
But in lemmy, it shows the same ‘WARN Error encountered while processing the incoming HTTP request: lemmy_server::root_span_builder: Unknown’ that I quoted in my first post
I don’t know what that error means. But I can see that lemmy still thinks it’s federating with 1/1 instances, and if I create a post on the test community, I can see it do:
INFO send:send:send_lemmy_activity: lemmy_apub::activities: Saving outgoing activity to queue https://f2d0-77-100-144-83.ngrok-free.app/activities/announce/create/ea41951a-6430-4e53-a5c4-afd644bf0824 Jan 11 11:47:43 lemmyA lemmy_server[977]: 2024-01-11T11:47:43.343193Z INFO send:send_lemmy_activity: lemmy_apub::activities: Saving outgoing activity to queue https://f2d0-77-100-144-83.ngrok-free.app/activities/create/bfcbf25e-37ab-4cfa-b44d-92cecacb1e56- To investigate the error, I tried sending JSONs in the same format that Mastodon uses, but utilising a command-line script.
The Follow works, but the Undo-Follow doesn’t.
The header returned shows the same error as journalctl does:
400 Bad Request "{\"error\":\"unknown\",\"message\":\"\"}"
I then recrafted the JSONs to be more like lemmy, changing the ‘id’ of Undo to have ‘activities/undo/UUID-string’, and that works. No entries in journalctl, and a 200 OK Response header.
- From lemmon.website, I can use the script to Follow and Unfollow communities on lemmy.ml
EDIT: I’ve had some success with this. Using the private key of the old Mastodon user I’m no longer getting stuff from ‘memes’ now, I think. (still stuff from ‘asklemmy’ though).
I’ll keep tinkering with this. Maybe during testing I followed asklemmy from another user. In the meantime though, it’d help enormously if any Follows from anyone at lemmon.website could be undone at your end (like I say: it’s all me)
(There is still the JSON problem though. It suggests that any ‘Undo-Follows’ you get from mastodon accounts aren’t being processed properly, so you’re maybe wasting subsequent federation activity. I realize that’s more a lemmy software issue than lemmy.ml issue, but it’s my understanding that you’re the same people!)
Thanks for the thorough investigation and for suggesting activitypub.academy. I found the problem and opened a pull request.
Let me know if you find any other incompatibilities with Mastodon (or other platforms). For example I only found out some weeks ago that Mastodon federates reports in a different format from Lemmy (which is supported from Lemmy 0.19.2).
-
