I did something similar to what you’re describing recently using Wireguard, and it’s working pretty well. The only difference in my setup is that I didn’t want to route internet traffic through the VPS, just get access to my local network (Site-to-Site VPN). I did accidentally route all of my traffic through my Wireguard VPS at one point during set up though, which seems promising based on what you want to do.

If you’re comfortable with using Docker, I used this guide from linuxserver.io to get setup: https://docs.linuxserver.io/images/docker-wireguard

linuxserver.io refers to terms such as “client” and “server” which don’t actually exist in Wireguard (all devices are just peers), but it’s useful as a way to visualize how you’d like to set everything up. I’m thinking all you’d have to do is setup your VPS in “Server” mode and define your peers, I don’t remember having to do anything special to get the routing to the internet set up.

The other part of the equation is setting up Wireguard on the target device. If you’re using an Android device, I can attest that the Wireguard app works great and is easy to setup. I unfortunately haven’t tried setting it up on a PC or laptop yet, but it looks like you could use the same linuxserver.io docker image in “Client” mode to get connected if you wanted to.