Hello everyone! During one of those illuminated evenings, I got the idea to move my small server in Scaleway to some more powerful server in Hetzner. If I will make the move, I am thinking of splitting the server in various VMs, to host different services that belongs to different trust boundaries, for example:

  • A Lemmy/writefreely instance
  • Vaultwarden/Gitea
  • Wireguard tunnel to my home infrastructure
  • Blogs, and other convenience services

In order to achieve the best level of separation, I was thinking of using VMs. My default choice would be Proxmox, because I used it in the past, and because I generally trust it, however I am trying to evaluate multiple options, and maybe someone has good or better experiences to share.

Other options I thought about are:

  • Run everything in Docker. I am going to do this nevertheless, but Docker escapes are always possible, especially with public facing images that I did not write myself and/or that require a host volume.
  • KVM directly? I am OK even without a GUI to be honest. I am not aware if there is some ansible module or even better Terraform provider for this, it would be great. (EDIT: I found https://registry.terraform.io/providers/dmacvicar/libvirt/0.7.1 which seems awesome!)
  • ESxi? I have no experience with this solution.

Any idea or recommendation?

  • @vegetaaaaaaa
    link
    English
    1
    edit-2
    1 year ago

    Ansible can be declarative if you do it right, and take the time to write a few roles to manage your use case. For example my ansible libvirt config looks like this:

    libvirt_vms:
      - name: front.example.org
        xml_file: '{{ playbook_dir }}/data/libvirt/front.example.org.xml'
        autostart: no
      - name: home.example.org
        xml_file: "{{ playbook_dir }}/data/libvirt/home.example.org.xml"
        state: running
    
    libvirt_port_forwards:
      - vm_name: front.example.org
        vm_ip: 10.10.10.225
        vm_bridge: virbr1
        dnat:
          - host_interface: eth0
            host_port: 22225 # SSH
            vm_port: 22
          - host_interface: eth0
            host_port: 19225 # netdata
            vm_port: 19999
    
    libvirt_networks:
      - name: home
        mac_address: "52:52:10:ae:0c:cd"
        forward_dev: "eth0"
        bridge_name: "virbr1"
        ip_address: "10.10.10.1"
        netmask: "255.255.255.0"
        autostart: yes
        state: active
    

    This is the only config I ever touch since the role handles changing configuration, running/stopping VMs, networks, etc. transparently. For initial provisioning I have a shell script that wraps around virsh/virt-install/virt-sysprep to setup a new VM in ~1 minute (It uses a preseed file, which is similar to what cloud-init offers). This part could be better integrated with ansible. Terraform has other advanced features such as managing hosts on cloud providers, but I don’t need those at the moment. If I ever do, I think I would still use ansible to run terraform deployments [1]

    Edit: the libvirt role if you’re curious