So I recently (a couple months ago) moved my fragmented docker-on-raspberry-pi architecture over to a Proxmox cluster. I’m running it on a pair of HP DL360 G6s, and I couldn’t be happier.
Except, well, I could be happier with just one more thing: high availability.
In particular, I want HA for my OPNsense firewall/router, but eventually for more of the workloads my family are depending on for life in general - Home Assistant, Plex, Overseerr, Immich, etc etc.
My current storage setup is a couple ratty old ARM-based NASes - an ancient Netgear ReadyNAS and an even more ancient Qnap TS-410. They’re both populated with 4 x 4TB (max raw size they can take) using RAID5, so I get about 22TB usable across the pair of them. They mostly store media for my Plex setup, but also support my 2N+C backup strategy for stuff like Immich, Paperless, and other important data.
My high-level plan is to grab another DL360, so I can have a quorum, then introduce a new storage system that:
- provides an iSCSI target for my Proxmox cluster; and
- can eventually grow to replace my old NASes.
The two solutions I’m pondering are:
- Build a TrueNAS setup from scratch - mini ITX case, board - the lot
- Pickup something tried, true and proven in the market, like a Synology
Up front cost is a consideration - I have a family to feed, so I can’t just run out and buy an 8-bay enclosure and fully populate it with 16TB disks.
Whatever I get, I’m likely to want to start with, say, 3 disks and grow it over time.
So, I guess this is a call out to the community to share any and all successes, war stories, and other advice. The more technical, the better. I want to make a sound, data-based decision here, and anecdotes from others who think like me are the best way to set my compass.
Cheers for anything you can offer!
One advantage of a separate TrueNas is that I run Proxmox Backup as a VM on the NAS. It’s entirely separate and obviously has access to my storage.
As do I. Not sure if it’s the right approach, but I run PBS in a CT on each node, backing up the other node’s guests.
But, with shared storage, I could get away with just the one PBS, correct?
One more option to consider would be doing HA with shared ZFS storage. Basically, if you have a zpool on each system with the same name and add it as shared storage, you can set up replication and have HA run off that. It’s a pretty simple setup and you can get by with just a couple extra disks per node rather than a whole separate machine.
Ah, interesting. Each node has a couple HDD slots free. I could populate those without introducing them to the hardware RAID set, and use ZFS on them instead? That could definitely be a good place for me to start. Thanks!
High availability storage is what the Ceph integration is for.
Edit: though it’s kind of a pain to set up and probably way overkill. A separate TrueNAS or similar appliance with a 10 gig uplink will be easier and probably just as reliable for your use.
I did look at Ceph as well, but balked at the 10gig requirement. I only have a couple of SFP slots free on my switch, which otherwise only provides 1Gbps sockets. Could be a future upgrade path, though, if I want to get to play with full HCI.
