cross-posted from: https://lemmy.ml/post/23615167

For better visibility I cross-post my question in this community.

Heyha ! I just came across a very odd issue/bug that somehow resolved by itself without knowing who or what was the culprit.

For context, YouTube doing his thing making nearly all public instances obsolete, I’m self-hosting a Piped instance in my homelab via Docker.

Everything is going smoothly, self-signed certs, traefik, accessible via Wireguard outside of my network, and and and !! LibreTube connects without any issues to my Piped instance on my Android phone and so does RiMusic.

However, in RiMusic when I was trying to access my synced Piped playlists, RiMusic went crazy and my playlist seemed to be in a query loop were I was unable to play any songs and was flickering alot.

  • Reboot the phone => Same behavior
  • Reboot the piped instance => Same behavior
  • Uninstall RiMusic/New docker piped instance => Same behavior
  • Flush everything from cache/playlist/configuration/data… => Same behavior

Nothing seems to resolve the issue software wise, next step check the logs (Interesting part):

My piped-nginx showed A HUGE amount of requests coming from my phone when accessing a Piped playlist:

"GET /playlists/d0e2c698-f3f4-435f-b2c9-96c6d3a88781 HTTP/1.1" 200 4161 "-" "ktor-client" "10.XXX.XXX.XXX"

Traefik also showed a lot of loadbalacing debug notifications something that never happens, because I’m the only user in my homelab setup !

My first though was that this is probably a RiMusic bug, but before reporting a report to GitHub, I did other debugging stuff.

  • Create an account and connect to a public piped instance
  • Create playlist/add some songs
  • Connect with RiMusic

The exact same behavior EXCEPT it stopped the loop after a few requests and made RiMusic usable again and was able to play my playlist without issues. Try again on my own instance but again, infinite loop, a lot of requests on Traefik and Piped-nginx. It even broke my Piped instance…

The only logical explanation is that the public piped instances have some request rate limiting (Yeah I know this is common practice and even mandatory on public instances). So here I go rate limiting my own requests to see if this could work as a temporary workaround while writing a GitHub bug report to RiMusic.

Adding some basic traefik labels just to give it a try:

labels:
  - "traefik.http.middlewares.test-ratelimit.ratelimit.average=10"
  - "traefik.http.middlewares.test-ratelimit.ratelimit.burst=20"

At first nothing happened but after a few docker compose -f down/up I was able to access my playlist from my own instance without any issues/bug/strangeness. Cool It works? So just out of curiosity I commented out the new traefik middelwares and restarted both container (Traefik/Piped). And … RiMusic playlist connected to my piped instance works without the ratelimite lines… Wait what ??

What just happend ? I have absolutely no idea… I don’t even know if the mentioned labels did anything… But everything works… No loading loop, No Traefik container overflown with loadblancer logs, No Piped-nginx with thousand request… It just vanished as it never existed in the first place.

I’m totally clueless except that somehow when accessing a playlist in private or public piped instance with RiMusic my phone went crazy with an infinite loop of api requests (Dunno if that’s the correct term :/). Here Am I with no idea what actually happend…

And yes my phone is Heavely debloated and firewalled (Magisk,rethinkDNS) so those are not unknown requests from the web or any open source application, whats so ever !


Sorry for the long write up I hope It’s readable and comprehensible. I just wanted to share my experience with you and If you also encountered some strange and inexplicable bug/issue that resolved by itself, feel free to share :).

PS: If someone has any good lead on what happened or some good insight where I should look next to get more out of this experience, I’m open to every good read !

  • Possibly linux
    link
    fedilink
    English
    113 hours ago

    I’ve seen some pretty crazy things in production level business systems.