Hello all, as the title says I’m looking for some advice / equipment recommendations or options. Little bit of a read to give some background and future information to hopefully get a good conversation going and learn some things along the way.
Me Aspiring networker. I currently work for an ISP as a field tech and have been for a while. Currently in the beginning stages of course work for a CCNA. Meaning I’m learning and not tech illiterate but there is a lot I don’t know and trying to catch up.
The Equpment Currently I do not have a ton for my “homelab”. 1 daily driver rig that is my bread and butter (decently high end), a middle of the road media server build that serves my home and about 5 others outside my network, but never at the same time (basically a mini ATX build with older but still relevant parts), and 21 other “various” devices from smart plugs to laptops). The rig, the server and my streamer are all hardwired, everything else is wireless. All connected via an Amplifi Gamers Edition router (ISP supplied modem, perfect working order, all new coax and signal levels to it are wonderful and no SNR issues). Subscribed speed is currently 600/20 Mb/s. My home is approximately 1900 sq ft including a partially finished basement. This is where the main portion of the router is at, with the rig and server. Both mesh points are upstairs.
The Problem I’ve come to the conclusion that my router cannot handle the traffic, and even with all the devices, I never come close to the top end of my subscribed speeds, even when the server is downloading as I have limiters in place. Random times even when the server is off, such as a few min ago, everything seemed to grind to a halt. Even my Sonos stopped playing music while trying to DL a file via mozilla at 340 kb/s.
The Plea for Help My cry for help is asking for a solution that will cover my entire home, but allow for me to continue and build on it as my homelab grows. My next wish is to build a firewall (maybe with opensense or PFsense). I would LIKE to have 6E availability but not a requirement. I’m not opposed to building something as all I have, I built. I do wish to start self hosting more and more as my learning journey continues and I attempt to automate my home as well.
Pretty much any suggestion is welcome and if there are any questions, please ask as well.
If a simple link for guidance is all that is necessary to guide me down a path, I will take it. I do not need to be spoon fed.
While opnSense is a fine solution (use it myself), I am not sure if your current router is actually the problem.
I suspect that the upload speed may be the culprit. Even when you are just downloading stuff there is data going bi-directionally. Even if it’s just TCP ACK packets. If your upload is fully saturated, download speed will suffer as well or even cut out entirely. You may also consider that Windows / Steam / Epic Games etc. like to download updates in the background, which might also fill up the capacity of your bandwith.
Does your router provide some kind of statistics for current bandwith use towards the internet? What does it show when those halts occur?
As for your planned homelab. On a birds eye level I would recommend using a small cheap computer with two network interfaces and opnSense as a router / firewall and Ubiquiti Unifi Access Points with the unifi controller software hosted on your media server. I can go into some more depth if you wish, but the official documentation for opnSense is pretty good.
I doubt it is getting fully saturated, not to say it’s not possible though. I can’t wait for our area to get High Split for symmetrical speeds. At the end of the day even if the router isn’t my issue, and I’m being saturated, I would like to move myself in a more secure and useable direction. Such as for home automation, additional servers and firewalls, and potentially getting into cyber security.
As for your planned homelab. On a birds eye level I would recommend using a small cheap computer with two network interfaces and opnSense as a router / firewall and Ubiquiti Unifi Access Points with the unifi controller software hosted on your media server. I can go into some more depth if you wish, but the official documentation for opnSense is pretty good.> This was basically gibberish to me, if you are willing for more details, I am more than happy to soak it in, and provide more questions following because that’s how it goes in the beginning right? lol
Thank you for your time as well. Reddit side was much less responsive than here. Judgy almost.
No problem, happy to help. Since I am not sure about your knowledge-level I will start from the basics, namely your ISP router.
It is in fact several devices in one:
-
A modem: The Modulator / Demodulator converts the signal from whatever standard they use on the cable net (likely some version of DOCSIS) into something the rest of your network can understand.
-
A router: This routes network packets between different networks. In your case that would be your internal network (probably 192.168.X.X) and the internet. Everytime different networks need to talk to each other a router will be needed between them. It also does NAT which is explained well here.
-
A firewall: While a router can guide a packet to where it needs to go, it can not decide whether it is welcome there. A firewall can check packets according to various configured criteria and decide to pass or block it. Very helpful to keep the FSB out of your porn collection.
-
A switch: Functions as a sort of central interconnect between wired devices. Much like an old telephone exchange where ladies would literally connect plugs with wires to create a direct connection between callers. The one in your ISP provided device is likely an “L2 dumb switch” <- more on this later.
-
A wifi access point: Very simplified basically a switch that works with radio waves instead of physical wires. Again, very simplified.
This is all very nice and comfortable for your basic home user. But it also greatly limits your options if you want to do something different than whatever the manufacturer of your ISP-device deems fitting. The most common option to cirumvent this is to separate the functions out into their own devices, using software and hardware that you have much more control over. Businesses do the same, depending on their size and requirements. It is therefore also good practice for a future career in system administration and networking.
In your case I would recommend the following:
- Check if your ISP-device allows you to set it in passthrough mode. This switches functions 2-5 off and turns it into a pure Modem and nothing else. DOCSIS comes in from the cable net, good old standard TCP/IP (among other things) comes out of the other. This is where you plug in device number 2/3.
2/3. Here is where opnSense comes in. It is a combined router AND firewall. This specific combination of functions is what most small and medium businesses use. Very large organisations may further separate out routing and firewalling into dedicated devices. But that is Facebook / Google / Microsoft level stuff. You connect the modem into the WAN-port of opnSense and device number 4 into the LAN-port. opnSense sits in the middle.
-
Get yourself a separate switch with as many ports as you need. Make sure that it supports the following functions: LACP (also known as IEEE 802.3ad), 802.1x and L2 VLAN (also known as IEEE 802.1q). POE might be useful as well, to supply some devices with power directly over the network. These weird words will become relevant later if you choose to go deeper into this rabbit hole. You won’t need them in the beginning but this way you are ready to really get into the deeper stuff.
-
Add in an access point to cover all your wifi needs. Make sure that it is an access point, not a wifi router. Some routers can be set to access point mode but thats just a waste of capabilities. The access point basically converts wired signals into radio. I recommend ubiquiti for this.
But for your next step you should start researching the five different functions I outlined in my first list. If I’m not sure about somethings (happens quite often ;-)) I literally type “whatever for dummies” into google and start reading. Don’t be embarassed to start at the bottom, we all came from there in the beginning. There are also some good youtube channels that might help. Once you are somewhat confident that you at least generally know what those magic five do you can start with the second list.
Feel free to ask further questions but it’s night time in my time zone so answers will take a while. Good luck and above all: Have fun. This is a fascinating topic and you can spend years exploring it.
I will be rewlading, researching, and contemplating on this for a moment. I will be back to you eventually. Withing the next couple days. I hope this post helps others now and in the future as well.
A lot of it makes sense. I do have a decent grasp on a concepts, but not specifics as you outlined. Honestly if I don’t know what docsis is, my employer would fire me 😂😂😂
But seriously, more questions to follow probably. 🫠
-
I’m not familiar with the Amplifi system but a quick look shows it has some QOS features to minimize “gaming” latency. I’m not sure how aggressive that is, maybe try turning it off?
Are you experiencing these slow downs over a wired connection? Lastly, you should be able to connect a computer directly to your cable modem and run some speed tests. That would give you an accurate idea of the current network performance.
One more thing, if you are working on your CCNA why not grab wireshark and record some packets? It’s a complex tool and I’m no expert but the latest versions color code traffic and can help point you towards an issue.
No idea about the Wireshark yet, as stated, beginnings lol. Haven’t gotten that far.
As for the latency settings, any device set to “latency/gaming” vs throughput is limited to 100mbs via the router, obv more than enough for basically anything except massive downloads. No option for settings like this if wired.
Both the rig and the server I have done speed tests with and the rig gets less DL but the same UL. The server is behind a VPN and the rig still gets less. Not a huge concern it’s everything else that slows down, including when plugged in.
Get a Qotom or any $200 4 RJ45 jack mini pc from Amazon and run Opnsense. Use a WAP for 6E if you’d like. That’s it
You had my curiosity, now you have my attention.
I’d be happy to try and answer any questions you might have, if I can. Cheers
Bad thing to say, I basically always have questions lol.
-
Reviewing the Qotom, there are many options, for future proofing myself, how much storage space is really needed (let’s factor in I plan to continue building my homelab.
-
For the WAP, is it possible to create a mesh system from more than one? There are only a few devices that I have “currently” that need to be hardwired but will obv. change in the future as I grow. (plus I’m sure the wife and guests will want wifi too gosh so neeedy
-
Storage space to run OPNsense? I think 128GB-256GB should be enough, might even be overkill (SSD/M.2 drives, don’t use it on an HDD). However, this question can become more nuanced if you want your hardware to be somewhat resilient to failures, like RAID. Also, if you’re going to run additional software like a reverse-proxy on your router + CA, I’d suggest more storage. I would personally use 512GB for a vanilla OPNSense install but as you can imagine that is extremely overkill and you can go years on a decent M.2 with good MBTF.
-
No. Not on OPNSense, AFAIK. If there is a way to create a mesh with third-party appliances, then maybe. Usually mesh WiFi comes under the topics of SDN, and I doubt OPNSense supports that (it might technically support parts of SDN since the definition is very broad, but YMMV). Usually people running Homelabs run UniFi/ASUS for Mesh Wifi (but I don’t trust companies for my privacy, don’t take my word for it).
Cheers
-
-
Build a router
