I’m looking to upgrade some of my internal systems to 10 gigabit, and seeing some patchy/conflicting/outdated info. Does anyone have any experience with local fiber? This would be entirely isolated to within my LAN, to enable faster access to my fileserver.
Current existing hardware:
- MikroTik CSS326-24G-2S+RM, featuring 2 SFP+ ports capable of 10GbE
- File server with a consumer-grade desktop PC motherboard. I have multiple options for this one going forward, but all will have at least 1 open PCIe x4+ slot
- This file server already has an LSI SAS x8 card connected to an external DAS
- Additional consumer-grade desktop PC, also featuring an open PCIe x4 slot.
- Physical access to run a fiber cable through the ceiling/walls
My primary goal is to have these connected as fast as possible to each other, while also allowing access to the rest of the LAN. I’m reluctant to use Cat6a (which is what these are currently using) due to reports of excessive heat and instability from the SFP+ modules.
As such, I’m willing to run some fiber cables. Here is my current plan, mostly sourced from FS:
- 2x Supermicro AOC-STGN-i2S / AOC-STGN-i1S (sourced from eBay)
- 2x Intel E10GSFPSR Compatible 10GBASE-SR SFP+ 850nm 300m DOM Duplex LC/UPC MMF Optical Transceiver Module (FS P/N: SFP-10GSR-85 for the NIC side)
- 2x Ubiquiti UF-MM-10G Compatible 10GBASE-SR SFP+ 850nm 300m DOM Duplex LC/UPC MMF Optical Transceiver Module (FS P/N: SFP-10GSR-85, for the switch side)
- 2x 15m (49ft) Fiber Patch Cable, LC UPC to LC UPC, Duplex, 2 Fibers, Multimode (OM4), Riser (OFNR), 2.0mm, Tight-Buffered, Aqua (FS P/N: OM4LCDX)
I know the cards are x8, but it seems that’s only needed to max out both ports. I will only be using one port on each card.
Are fiber keystone jacks/couplers (FS P/N: KJ-OM4LCDX) a bad idea?
Am I missing something completely? Are these even compatible with each other? I chose Ubiquti for the switch SFP+ since Mikrotik doesn’t vendor-lock, AFAICT.
Location: US
Unfortunately, the situation is not so simple. Even if the various vendor-locks weren’t a thing, the fact is that the testing matrix of xcvrs on the market crossed with the number of switch manufacturers and all their models is ginornous, and it would be a herculean effort to acquire, let alone validate even a subset of all combinations.
While SFP is defined in a standard, the allowable variances – due to things like manufacturing capabilities and the realities of environmental influences – mean that it’s possible for two compliant transceivers to just not link up. It’s unfortunate, but interoperability with so many players and at such cut-throat margins leads to this reality.
And since it’s a chain of components, any incompatibility of switch, xcvr, or fibre can wreck a link, and then the blame game hot-potato gets tossed around since no vendor wants to investigate a link issue if it might not be their fault.
In my experience, though, the initial link negotiation is the most problematic part when building a link that isn’t all supplied by one vendor. Once past this, I find that a link rarely has issues thereafter. Which is good if you’re able to return xcvrs if they don’t work for your setup.
This is just anecdotal, but I have never once experienced an issue with SFP+ vendor lock. I have mix-and-matched transceivers between Mikrotik, DLink, TPLink, Dell Enterprise, and Xyxel switches as well as both Mellanox and Intel NICs. The only issue i can recall is some auto-negotiatiin issues using 1GB modules in a Mellanox switch. Manually setting the link rate fixed it. I use a combination of 10Gb fiber, 10Gb copper, and 1Gb copper modules as well as DAC depending on the situation.
I know that vendor lock does exist, but it’s not as widespread with modern hardware.
I think you’re right, as prosumer and low-end enterprise switch vendors have less of an incentive to bundle first-party xcvrs along with switch sales. However, the ISP and large-enteprise market segments still have vendor locks, although many have an “allow unsupported xcvr” mode which will apply best-effort to operate a third-party xcvr but the warranty won’t be honored while such a xcvr is installed.
The likes of Cisco and HPE do things like this, but given that the target customers of such switches are buying them in the hundreds to thousands, and each switch already costs thousands of dollars, the cost of first-party pluggables is just a part of the deal. Such customers also value reliability to a greater degree, so even a miniscule prospect of incompatibility will be avoided.
Insofar as it pertains to this community, the ability to enable the unsupported xcvr mode means old high-end equipment gets a second life in someone’s homelab, since warranties stop mattering there