cross-posted from: https://lemmy.ml/post/23615167

For better visibility I cross-post my question in this community.

Heyha ! I just came across a very odd issue/bug that somehow resolved by itself without knowing who or what was the culprit.

For context, YouTube doing his thing making nearly all public instances obsolete, I’m self-hosting a Piped instance in my homelab via Docker.

Everything is going smoothly, self-signed certs, traefik, accessible via Wireguard outside of my network, and and and !! LibreTube connects without any issues to my Piped instance on my Android phone and so does RiMusic.

However, in RiMusic when I was trying to access my synced Piped playlists, RiMusic went crazy and my playlist seemed to be in a query loop were I was unable to play any songs and was flickering alot.

  • Reboot the phone => Same behavior
  • Reboot the piped instance => Same behavior
  • Uninstall RiMusic/New docker piped instance => Same behavior
  • Flush everything from cache/playlist/configuration/data… => Same behavior

Nothing seems to resolve the issue software wise, next step check the logs (Interesting part):

My piped-nginx showed A HUGE amount of requests coming from my phone when accessing a Piped playlist:

"GET /playlists/d0e2c698-f3f4-435f-b2c9-96c6d3a88781 HTTP/1.1" 200 4161 "-" "ktor-client" "10.XXX.XXX.XXX"

Traefik also showed a lot of loadbalacing debug notifications something that never happens, because I’m the only user in my homelab setup !

My first though was that this is probably a RiMusic bug, but before reporting a report to GitHub, I did other debugging stuff.

  • Create an account and connect to a public piped instance
  • Create playlist/add some songs
  • Connect with RiMusic

The exact same behavior EXCEPT it stopped the loop after a few requests and made RiMusic usable again and was able to play my playlist without issues. Try again on my own instance but again, infinite loop, a lot of requests on Traefik and Piped-nginx. It even broke my Piped instance…

The only logical explanation is that the public piped instances have some request rate limiting (Yeah I know this is common practice and even mandatory on public instances). So here I go rate limiting my own requests to see if this could work as a temporary workaround while writing a GitHub bug report to RiMusic.

Adding some basic traefik labels just to give it a try:

labels:
  - "traefik.http.middlewares.test-ratelimit.ratelimit.average=10"
  - "traefik.http.middlewares.test-ratelimit.ratelimit.burst=20"

At first nothing happened but after a few docker compose -f down/up I was able to access my playlist from my own instance without any issues/bug/strangeness. Cool It works? So just out of curiosity I commented out the new traefik middelwares and restarted both container (Traefik/Piped). And … RiMusic playlist connected to my piped instance works without the ratelimite lines… Wait what ??

What just happend ? I have absolutely no idea… I don’t even know if the mentioned labels did anything… But everything works… No loading loop, No Traefik container overflown with loadblancer logs, No Piped-nginx with thousand request… It just vanished as it never existed in the first place.

I’m totally clueless except that somehow when accessing a playlist in private or public piped instance with RiMusic my phone went crazy with an infinite loop of api requests (Dunno if that’s the correct term :/). Here Am I with no idea what actually happend…

And yes my phone is Heavely debloated and firewalled (Magisk,rethinkDNS) so those are not unknown requests from the web or any open source application, whats so ever !


Sorry for the long write up I hope It’s readable and comprehensible. I just wanted to share my experience with you and If you also encountered some strange and inexplicable bug/issue that resolved by itself, feel free to share :).

PS: If someone has any good lead on what happened or some good insight where I should look next to get more out of this experience, I’m open to every good read !

  • @[email protected]
    link
    fedilink
    English
    37 hours ago

    Sounds like some kind of caching issue on your device, as in it probably partially loaded something and got stuck trying to refresh stale data or something.

    Most of these weird issues tend to disappear after clearing caches (e.g. deleting user data from your phone and logging in again). Rate limiting may have fixed it by giving an error code instead of giving the same success, which could have triggered a cache invalidation.

    So if it happens again, that’s what I’d try. That’s my best guess without trying it myself.

  • Possibly linux
    link
    fedilink
    English
    19 hours ago

    I’ve seen some pretty crazy things in production level business systems.

  • @[email protected]
    link
    fedilink
    English
    413 hours ago

    Not so much self-hosting related (though I do run a Jellyfin server on this PC), but within the past week, I was sure I was gonna have to reinstall my os (and probably take the opportunity to jump distros) because audio just would not work. Everything seemed to think it was fine. Even OBS was picking up the audio, but there was no output. Persisted through reboots and everything.

    I have no idea what happened to cause it or fix it.

    • @AtariDump
      link
      English
      16 hours ago

      I have this issue with a dual boot windows/linux mint machine. Happens when I boot into windows then go back to mint; mint has no sound.

      Easiest fix I’ve found is to disable the onboard audio in the BIOS, boot into mint, shutdown, re enable audio in the BIOS, boot into Mint. The windows kernel seems to hold on to the audio device for some reason.

    • @[email protected]
      link
      fedilink
      English
      211 hours ago

      I did reinstall my os (and hop over to arch while I was at it) because I had a bug with Forza that made it crash on login.

      It didn’t help (I think it was a bug with the save file) but at least I can say I installed arch the hard way