Mooching off this other post
Primary question: What do people do for their reverse proxies (and associated ACME clients)? Do you have a single unified one? Or do you use separate proxies for each stack? Or some mess in between?
My use case question: For example, I have a (mess that is a) Nextcloud instance with a separate stack with nginx and ACME, a SearXng that wants to run caddy (but has shoved into the nginx).
But now I have a Lemmy docker that has a custom(?) nginx instance, should I just port it to my existing nginx or run them side by side?
I use a single unified traefik to front all of my services, no matter how they ship. Despite the slight overhead, it’s closer to a truly idempotent architecture. I’ve unfortunately had to test that twice now in my selfhosting career.
Traefik is very solid and I’ve had very few issues with it I didn’t self inflict. Documentation is very thorough.
I also run Traefik and it never stutters. But getting it set up at ALL was a chore. I tried four times and failed. Each time I spent several days full time on it. It’s not that I skipped the docs, I actually am a RTFM kinda guy. But too much was implied in the docs and I never really felt like I knew why I was doing stuff. At least for me it was harder to set up than Nextcloud, Jellyfin, Gitea, Resilio and Vaultwarden COMBINED.
Some settings only work in a static config file, others in a “dynamic” config file and then there is container-specific labels too. It all needs to fit in with each other and error mesages were of course hidden away in docker logs. You can attach labels to containers with and without escaping them, and choosing wrong sends you down several rabbit holes at once. The config structure is probably intuitive to Go devs, but that really ain’t me. Oh, there’s also 3 different but equal formats for conf files too.
I read countless guides and it finally worked on attempt 5. All just because I liked the autoconf for all containers. I could have been done with reverse proxies within a day had I just chosen a different one.
Now I am even debating wether I should keep it at all, because I’d rather not mount the docker sock into my reverse proxy, the one software that ultimately connects to the web directly.
I’d rather not mount the docker sock into my reverse proxy,
You don’t have to if you use the dynamic file config. I’ve mentioned this before and debated to the ends of earth for even suggesting such a thing. But it all aspects is dynamic file configuration better.
Of you use IaC in your set up, it gets even easier because then you can just set up templates that automatically create file configs and add them to your reverse proxy seamlessly.
Right now with one Terraform apply, I create my docker container, traefik config and my homepage service.
I’ll have to look into using dynamic file configuration more heavily then. But how do you personally set up networking, if traefik doesn’t handle it all for you? Can you just tell traefik the container name from docker-compose in the dynamic file config? Also, what is IaC? Cause it sounds great.
IaC stands for infrastructure as code.
Traefik’s dynamic file config just scans a folder for .yaml files. So i just generate a file automatically with the container’s network address and copy the file into traefik’s folder.
Traefik just looks for the container address in the file (either IP or container name). The files are basically the yaml version of the docker label.
Yeah, this experience with Traefik lines up pretty well with mine. It can be a steep learning, and the fact that half the search results out there are for Traefik v1 (with a completely different configuration syntax to v2) doesn’t help. But once it’s up and running, the dynamic configuration based on container labels is pretty darned nice.
Now I am even debating wether I should keep it at all, because I’d rather not mount the docker sock into my reverse proxy, the one software that ultimately connects to the web directly.
You could switch to Podman, in which case you’d give it a non-root, read-only socket that isn’t the keys to the kingdom. Or maybe rootless Docker would be an easier switch and still give you some of those benefits.
Podman is on my todo list! I like the ideas behind Podman and because I am already familiar with docker containers, I hope that I can transfer most of my stuff over almost pain free. But I heard the linuxserver.io images are unsupported on podman/rootless docker and might give me trouble. We’ll see!
On the other hand, I have recently fallen in love with NixOS and would love to consolidate on a common Nix config for all my servers, Raspberries and maybe eventually desktops. It’s the perfect time to try out podman!
I don’t know about pain-free. :D See my earlier post on the topic: https://lemmy.world/post/213870
But it might be worth it anyway depending on your needs! Trying out NixOS sounds cool though… I’ve been meaning to look into that.
Do you have a guide for Traefik that you recommend?
I tried about 6 guides:
This is the definitive guide that always works even if you sub in cloudflare for let’s encrypt.
https://www.smarthomebeginner.com/traefik-docker-compose-guide-2022/
Thank you!
I use the built-in Traefik that ships with k3s. Works great for me – a bit of a learning curve, as I was really only familiar with Nginx, but now that I’m more used to it I’m a big fan.
Since nobody has responded to the ACME / Let’s Encrypt part of the question yet, I’ll chime in: I also use Traefik as a reverse proxy (and an ACME client), one unified instance per machine. (There are some exceptions, like for Mailu that requires its own nginx reverse proxy.) But for Let’s Encrypt, I recently switched from the TLS challenge to the DNS challenge. That required switching my DNS server from CoreDNS to PowerDNS, but thus far it seems totally worth it. Now I can easily get TLS certs for servers on my private network at home without opening them up to the internet for HTTP/TLS challenges.
Can you expand on more detail on how your TLS certs work? Looking to do similar.
I am just running the normal nginx image with
/etc/letsencrypt:/etc/ssl/privateas volume.certbotdoes the rest. If you need help with the exact config just search for relevant keywords, there are tons of good tutorialsHere’s an overview of the Let’s Encrypt DNS challenge type in case you haven’t seen it: https://letsencrypt.org/docs/challenge-types/#dns-01-challenge
Basically, when Traefik goes to request or renew a certificate, Let’s Encrypt tries to look up a special DNS record on your domain so you can prove that the request for the certificate is legit. To make that work, Traefik first hits your DNS provider via API and temporarily inserts that special record so it’s there when Let’s Encrypt performs the lookup for it. In my particular case, I’m using self-hosted PowerDNS and it’s built-in API (configured to only respond via a Wireguard tunnel). But you don’t have to self-host DNS for this to work… Traefik has a long list of supported providers: https://doc.traefik.io/traefik/https/acme/#dnschallenge
Oh this look great, thanks!
So there is no port forwarding needed? Works behind CG-NAT?
No port forwarding needed and works behind CG-NAT—assuming your DNS server is hosted elsewhere.
That’s great!
I run the HAProxy and ACME packages available from PFSense on my firewall.
Certificate rotation is automatic, connected to my domain in cloudflare and I have 1 *:443 listener on a virtual IP with about a dozen backends pointing directly to each app.
