Hello!
I’ve had Linux server/s at home for hobby and fun for years. My current “main” server is quite old, quite slow, and in need of an upgrade.
I acquired a used Thinkstation Mini SFF, which blows the old one out of the water, being like 15% of the size. The heaviest usage is a Plex server for my and my family. Current setup has worked well but it’s unable to transcode properly, without a lot of noise and only some codecs/resolutions.
I installed Proxmox on this Thinkstation device and it works very well. I migrated my WP family blog from a Virtualbox to Proxmox VM, worked perfectly.
I then used the helper scripts to install Plex for testing. I got hardware acceleration working with the i5-8500T/UHD 630. Did some testing and it seems to work great.
I have yet to migrate the server and the data. The Thinkstation has ~320gb SSD disk which is way to small for my video collection.
So I’m what I’m wondering, what is the best next step? I can one or more similar devices for cheap/nothing.
I don’t really understand the nodes function of Proxmox properly.
Should I:
A) Use my current one as a “main” node, install large SSD/HDD on 1 or 2 similar devices and connect them as storage nodes?
B) Add large USB SSD/HDD to my existing Proxmox device?
C) Setup a simple NAS device and connect Proxmox to that?
D) Something else?
After that’s decided, I could then move the Plex installation/setup from my current server to the Proxmox setup.
The usage is not heavy, 3 concurrent streams are rare. Media is currently mostly 1080p.
My main concern is the Plex setup, but if one setup is more futureproof, smarter, handier, cooler, nicer than the other, I’d greatly appreciate your input.
Strongly recommend setting up a NAS device to store your media - it’ll stop you from having to migrate data and reconfigure Plex when your storage fills up. You can upgrade the storage whenever you need and mount it via NFS/SMB in Proxmox under Datacenter > Storage
You can even run the NAS in Proxmox if you feel froggy. Just make sure to connect all the drives directly to your VM instead of the Proxmox Host, and make sure it’s started BEFORE you try to run any other VMs because those shares won’t be available without that VM being on.
You might want the NAS on other hardware to avoid that, but that’s your decision.
Thanks for the reply. Yes, sounds good.
Why not upgrade the disk in the new proxmox device?
Hi, thanks for the reply. Yeah, that is of course an possibility also, just backup the containers and vms and start over. But I’m still keen on the idea of adding another device, possibly for mirroring or other types of backup.
Your device only has 4 cores so I would go with a separate storage device.
I’d suggest using ZFS for the OS and the VM storage. If you have multiple nodes, you can set up replication easily for easy migration or recovery if your main node goes down. If you set up plex on a NAS, you would have to set up the NAS shares on the other node(s) in case you end up bringing the VM up on another node.
What I’ve done is have a very large ZFS striped-mirror that contains all my VMs and their attached drives, as you can specify and overprovision virtual drives on the same zpool. Then these VMs replicate to the other nodes every hour. I can also live migrate the VMs very quickly since they’re replicated constantly so if I need to upgrade a node, I batch migrate everyone over to the other nodes, do my upgrades and migrate everyone back and upgrade the other nodes, then rebalance them where I like them. I get that you’re on an SFF, so if you can get 2 drives in it and mirror them, you’re probably fine.
And then all that is backed up via Proxmox Backup Server on a shitty old i3 with a local attached SATA and other USB drives I periodically change out and take offsite for replication to another offline proxmox server.
You can also pass through a GPU for hardware encoding to your Plex/jellyfin VM.
Thanks for the input, sorry for the delayed answer.
I was looking into the replication feature, and it sounds cool. I can acquire used identical devices from work, which is handy for this.
These are my ideas now, feedback would be greatly appreciated:
A) Get another identical device, upgrade the internal storage to 1tb nvme and setup Proxmox. Move existing ct’s /vm’s to that while upgrading the storage on the other device to same kind of storage and reinstalling Proxmox, then joining them as nodes in a cluster with replication. Correct thinking?
B) Get another identical device, upgrade the internal storage to 1tb nvme and setup Proxmox Backup Server. Backup the ct’s / vm’s, upgrade storage on the the other device to same kind of storage and reinstalling Proxmox, then restoring the vm’s / ct’s and have it act as the main server.
In either case, there would be two identical devices, providing different types of backup.
I’d also add a third device, a simpler NAS with more storage for Plex and other stuff. This might or might not be backed up, as it’s not that important. I might even experiment with something like GlusterFS, since the main cost for me is the HDD/SDD storage space. Electricity is cheap for me and SFF don’t take up much space.
Any thoughts?
I’m certainly a proponent of having at least 2 nodes, even if one is offline most of the time and only brought up for replication and upgrades so I’d be inclined to go with A. But one thing to consider is that PM wants a quorum on node votes for when you’re doing cluster operations like joining and leaving nodes, changing storage, etc. For this purpose I have the PBS server act as the 3rd vote with a qdevice installed on it and joined to the cluster. You can also accomplish this by setting expected_votes=1 in a 2 node cluster, but I’ve found that a little unreliable and the qdevice method always works. You can also put the qdevice on a VM that you keep up and migrate between nodes when you’re upgrading to keep quorum, though I’ve never tried that. But the qdevice service can go on any linux system that you’re likely to have up all the time, raspberrypi, your desktop, etc.
You can add and remove nodes as you wish, I’ve done that in hardware upgrades where I’ve moved guests off to another node, removed that node from the cluster (preferably from the node you’re upgrading itself and make damn sure it’s replicated to the remaining node before you down the node), completely rebuilt the node and reinstalled it, then joined it back to the cluster and migrated guests back. Again, you will need a quorum active on the cluster throughout this process.
I like the idea of keeping the Plex storage off the PM server, though it’s not entirely necessary. If it’s a Linux device, you can also install a client that will backup the system and anything else you need to PBS, it will do non-PM device backups as well. I have my desktops, laptops and HTPC backup that way and have used it to recover and even reinstantiated a laptop backup as a VM when I needed to check something after I had the physical laptop crater as it was easier to restore it as a VM so I could use the DE to find what I needed than to sort it out of the filesystem. I’ve only every used it with ext4 installed systems, so use with caution on more exotic setups.
You might also have your plex storage connected to the plex VM via iSCSI, nbd or sshfs? Just a thought there. Personally, I wouldn’t trust samba sharing to wash my toilet, but plenty use that as well. NFS can be unreliable as well as it’s recovery from network outages isn’t perfect.
Proxmox also is integrated with CephFS, which might be of interest if you’re also thinking about gluster. I’ve never used it or really investigated how PM uses it, but it might fill your needs there.
