cross-posted from: https://reddthat.com/post/21668140

I have a VPN daemon that needs to run before the client will work. Normally, this would have been set up automatically by its install script, but the system is immutable.

I’ve created the systemd service via sysyemctl edit --force --full daemon.service with the following parameters:

[Unit] 
Description=Blah
After=network-online.target

[Service]
User=root
Group=root
ExecStart=/usr/bin/env /path/to/daemon

[Install]
WantedBy=multi-user.target

I’ve verified that the daemon is actually executable, and it runs fine when I manually call it via sudo daemon. When I try to run it with sudo systemctl enable --now daemon.service, it exits with error code 126.

What am I missing?

Edit: Typo, and added the relevant user and group to the Service section. Still throwing a 126.

Solution: the system wanted /usr/bin/env in ExecStart to launch the binary. The .service file above has been edited to show the working solution.

  • RustmilianEnglish
    1•5 months ago

    I assume so, but just to be sure, have you run sudo systemctl enable blah.service then reboot? It’ll symbolic link to the systemd auto start service and run it at boot.
    Also, make sure everything is marked as executable; especially whatever you have “/path/to/daemon” set as. sudo chmod +x /path/to/daemon
    Restart the service or reboot then :
    sudo systemctl status blah.service

    • @[email protected]OPEnglish
      2•5 months ago

      Yep, more specifically I tried sudo systemctl enable --now daemon.service. Gives the same error, and maybe that’s because it’s some kind of binary.

      sudo /bin/bash /path/to/daemon throws the same error, but sudo /path/to/daemon does not. However, if I drop , /bin/bash from the service file, it throws a 203 error instead.

      • RustmilianEnglish
        3•5 months ago

        Is the daemon a binary? If so drop the bash part and try sudo chmod 755 /path/to/daemon.

        • @[email protected]OPEnglish
          3•5 months ago

          Tried that, back to 203/Exec error. It’s like ExecStart wants me to specify a program to launch it, and bash clearly isn’t it.

          • RustmilianEnglish
            3•5 months ago

            Try ExecStart=/usr/bin/env /path/to/daemon
            Also what’s the output of ldd /path/to/daemon & sudo systemd-run /path/to/daemon ? Maybe check systemctl show-environment. Maybe try adding Type=simple , this tells systemd that the service will fork.

            If that fails, we could try ExecStart=/usr/bin/strace -f -o /tmp/daemon_strace.log /path/to/daemon for stactrace & ExecStart=/bin/sh -c '/path/to/daemon > /tmp/daemon.log 2>&1' to log the daemon.

            • @[email protected]OPEnglish
              3•5 months ago

              Omg, adding /usr/bin/env worked. Launched the daemon, and the client is able to launch and connect a WireGuard tunnel.

              systemctl show-environment lists /usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin on the PATH, so maybe that’s why that worked…? (I’m going to have to go read up on env).

              Either way, I did a reboot to verify, and it’s definitely running. Now I just need to tweak it a bit so it tries to reconnect if the network drops out, but holy shit, I appreciate the help.

              • RustmilianEnglish
                3•5 months ago

                Good to hear that it worked.
                To explain env, typically when systemd is running a service it only provides a very minimal environment. When using env it passes more of the environment variables and whatnot from userspace, so it’s likely that the binary daemon was looking for specific environment variables and it returned an empty string and that’s what caused error, it’s also useful if the daemon’s location changes during runtime or if it’s not in a standard location.