10G or faster networking options.¶
Want connectivity faster then 10G?
Well- I have a list of NICs, switches, and notes.
Info
This content is NOT sponsored.
I do personally own these switch models mentioned below: - Unifi USW-Aggregation - Mikrotik CRS504-4XQ - Mikrotik CSS610-8G-2S+ - Brocade ICX6610-48P - Brocade ICX6610-48 - Brocade ICX6450-24P
These switches were purchased by myself, with no undisclosed benefits.
This post DOES include affiliate links. More information at the bottom of this post.
Network Adapters¶
Note- price estimates last updated Dec 18, 2024.
I do not recommend picking up the 10G NICs. Instead, Spend 5 or 10$ more, and pick up one of the newer 25G NICs. They are compatible with 10G, but- are an entire generation newer.
This is also not a complete list. This list will be updated as I discover new NICs worthy of being added.
Make | Model | Ports | Type | Multi-Gig | Speed | Generation | Price | Link | Notes |
---|---|---|---|---|---|---|---|---|---|
Chelsio | T520-CR | Dual | SFP+ | No | 10G | Chelsio T5 | $30-$70 | eBay Link | nan |
Chelsio | T540-CR | Quad | SFP+ | No | 10G | Chelsio T5 | $40-$100 | eBay Link | One of the few quad 10G NICs. |
Chelsio | T580-CR | Dual | QSFP+ | No | 40G | Chelsio T5 | $20-$60 | eBay Link | nan |
Chelsio | T6225-CR | Dual | SFP28 | No | 25G | Chelsio T6 | $50-$150 | eBay Link | nan |
Dell | 099GTM4 | Dual | SFP28 | No | 25G | 500 Series | $10-$20 | eBay Link | Compatible with many Dell servers (e.g., R730XD). Not picky with modules. 2x1G + 2x10G RJ45. Intel X540. |
Dell | 6VDPG5 | Quad | SFP+ | No | 2x 1G + 2x 10G | 700 Series | $20-30 | eBay Link | I do NOT recommend getting this NIC. It is extremely picky with modules. |
Dell | R887V1 | Quad | RJ453 | No | 2x 1G + 2x 10G | ConnectX-4 | $10-$20 | eBay Link | Compatible with many Dell servers (e.g., R730XD). Not picky with modules. |
Intel | E810-CQDA2 | Dual | QSFP28 | No | 100G | 800 Series | $260+ | eBay Link | nan |
Intel | E810-CQDA2 | Single | QSFP28 | No | 100G | 800 Series | $300+ | eBay Link | Oddly, the single port version costs more then the dual-port version. |
Intel | E810-XXVDA2 | Dual | SFP28 | No | 25G | 800 Series | $150+ | eBay Link | nan |
Intel | E810-XXVDA4 | Quad | SFP28 | No | 25G | 800 Series | $200-300 | eBay Link | nan |
Intel | I226-T1 | Single | RJ45 | Yes* | 2.5G | nan | nan | eBay Link | 2.5G/1G Adapter. Not the best reviewed NICs. |
Intel | X540-T2 | Dual | RJ453 | No | 10G | 500 Series | $15-$30 | eBay Link | These work well when you need a copper 10G link. |
Intel | X550-T2 | Dual | RJ453 | Yes | 10G | 500 Series | $70-120 | eBay Link | nan |
Mellanox | CX314A | Dual | QSFP+ | No | 40G | ConnectX-3 | $10-$20 | eBay Link | Recommend newer NICs (e.g., ConnectX-4). Can support 10G with SFP -> QSFP adapter. |
Mellanox | CX354A | Dual | QSFP+ | No | 40G | ConnectX-3 | $10-$20 | eBay Link | Recommend newer NICs (e.g., ConnectX-4). Can support 10G with SFP -> QSFP adapter. |
Mellanox | CX4121A | Dual | SFP28 | No | 25G | ConnectX-4 | $30-$40 | eBay Link | nan |
Mellanox | CX416A | Dual | QSFP28 | No | 100G | ConnectX-4 | $100-$200 | eBay Link | This is the NIC I use in most of my servers. |
Mellanox | CX455 | Single | QSFP28 | No | 100G | ConnectX-4 | $100 | eBay Link | nan |
Mellanox | CX516A | Dual | QSFP28 | No | 100G | ConnectX-5 | $150-$300 | eBay Link | nan |
Mellanox | MCX311A-XCAT | Single | SFP+ | No | 10G | ConnectX-3 | $15-$25 | eBay Link | ASPM broken on newer Intel CPUs; works with mlx4_core . Recommended to choose 25G. |
Mellanox | MCX312B | Dual | SFP+ | No | 10G | ConnectX-3 | $30-$40 | eBay Link | Consider ConnectX-4 10/25G instead. |
Mellanox | MCX4111A | Single | SFP28 | No | 25G | ConnectX-4 | $20-$40 | eBay Link | nan |
TP-Link | TX201 | Single | RJ453 | Yes* | 2.5G | nan | 30$ | Amazon | 1/2.5G |
TP-Link | TX401 | Single | RJ453 | Yes | 10G | nan | 100$ | Amazon | 1/2.5/5/10G |
Notes¶
Please keep in mind, compatible speeds for a given network adapter.
25G NICs will nearly always also support 10G. (and slower)
100G NICs, can generally support anything less.
If you wish to run a 40/50/100G NIC at 10G, you will need a QSFP+ to SFP+ Adapter.
Only NICs which specify "Multi-Gig" are capable of 2.5G, or 5G. (TP-Link NICs & Intel i226 are the only NICs flagged at this time)
56Gbit/s is infiniband only, and only applies when running a Mellanox NIC, in IB mode, when connected to a supported infiniband switch.
Mellanox Infiniband NICs (Every one of the listed Mellanox NICs) can be ran in either ethernet mode (where they behave like a normal NIC), or Infiniband mode. I strongly recommend ETH mode. Guide: Switch Port Mode
I personally prefer Mellanox NICs. In my experiences, these are the best bang for the buck, are generally really cheap, plug and play, and have been extremely reliable.
In my experiences, Intel, Mellonax, and Chelsio have all been plug and play with both Linux and Windows.
Switches¶
Note, ALL switches listed, are MANAGED switches, with support for vlans.
Table¶
Notes: - Data used from manufacturer spec-sheets when possible. - Footnotes are included for additional context.
Manufacturer | Model | Layer 3 | Max Speed | RJ45 | SFP | SFP+ | QSFP+ | Multigig | Noise | Power | POE | Price | Link |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Brocade | ICX 6450-24 | Yes | 10G | 24 | - | 4 | - | No | 37.9 dBA | 20-37w | No | 100$ | eBay |
Brocade | ICX 6450-24P | Yes | 10G | 24 | - | 4 | - | No | 39.2 dBA | 21-400w9 | Yes | 100$ | eBay |
Brocade | ICX 6450-48 | Yes | 10G | 48 | - | 4 | - | No | 37.2 dBA | 30-55w | No | 60-100$ | eBay |
Brocade | ICX 6450-48P | Yes | 10G | 48 | - | 4 | - | No | 55.5 dBA | 31-776w9 | Yes | 60-100$ | eBay |
Brocade | ICX 6610-24 | Yes | 40G | 24 | - | 1610 | 210 | No | 39.6 - 48.7 dB | 120-140w | No | 100$ | eBay |
Brocade | ICX 6610-24P | Yes | 40G | 24 | - | 1610 | 210 | No | 39.6 - 48.7 dB | 120-140w | Yes | 100-140$ | eBay |
Brocade | ICX 6610-48 | Yes | 40G | 48 | - | 1610 | 210 | No | 39.6 - 48.7 dB | 165-185w | No | 60-100$ | eBay |
Brocade | ICX 6610-48P | Yes | 40G | 48 | - | 1610 | 210 | No | 39.6 - 48.7 dB | 165-185w | Yes | 60-100$ | eBay |
Brocade | ICX 7250-24 | Yes | 10G | 24 | - | 8 | - | No | 41.9 dBA | 39-57w | No | 75-120$ | eBay |
Brocade | ICX 7250-24P | Yes | 10G | 24 | - | 8 | - | No | 44.7 dBA | 48-454w9 | Yes | 75-120$ | eBay |
Brocade | ICX 7250-48 | Yes | 10G | 48 | - | 8 | - | No | 44.5 dBA | 50-70w | No | 90-140$ | eBay |
Brocade | ICX 7250-48P | Yes | 10G | 48 | - | 8 | - | No | 45.9 dBA | 70-942w9 | Yes | 90-140$ | eBay |
Brocade | ICX 7150-24 | Yes | 10G | 24 | - | 4 | - | No | Fanless | 14-24w | No | 150-300$ | eBay |
Brocade | ICX 7150-24P | Yes | 10G | 24 | - | 4 | - | No | 41.4 dBA | 32-472w9 | Yes | 150-300$ | eBay |
Brocade | ICX 7150-48 | Yes | 10G | 48 | - | 4 | - | No | Fanless | 24-39w | No | 250-600$ | eBay |
Brocade | ICX 7150-48P | Yes | 10G | 48 | - | 4 | - | No | 41.8 dBA | 47-491w9 | Yes | 250-600$ | eBay |
Mellanox | SX6036 | Yes | 40/56G7 | 2 | - | - | 36 | No | Unknown | ?? - 231w | No | 100-120$ | eBay |
Mellanox | SX1036 | Yes | 40/56G7 | 1 | - | - | 36 | No | Unknown | 78 - 100w? | No | 600$ ?? | eBay |
Mellanox | SX6012 | Yes | 40/56G7 | 1 | - | - | 12 | No | Unknown | 28w - ?? | No | eBay | nan |
Mellanox | SX6018 | Yes | 40/56G7 | 2 | - | - | 18 | No | Unknown | ?? | No | 100-250$ | eBay |
Mellanox | SX60056 | No | 56G IB Only | - | - | - | 12 | No | Unknown | ?? | No | eBay | nan |
Mellanox | SX60156 | No | 56G IB Only | - | - | - | 18 | No | Unknown | ?? | No | eBay | nan |
Mellanox | SX60256 | No | 56G IB Only | - | - | - | 36 | No | Unknown | ?? | No | eBay | nan |
Mikrotik | CRS304-4XG-IN | Yes**11 | 10G | 1 | - | 4 | - | Yes | Fanless | 7-21w | No | 199$ | Amazon |
Mikrotik | CRS305-1G-4S+IN | Yes**11 | 10G | 1 | - | 4 | - | No | Fanless | 10-18w | No | 120$ | Amazon |
Mikrotik | CRS309-1G-8S+IN | Yes**11 | 10G | 1 | - | 8 | - | No | Fanless | 17-23w | No | 269$ | Amazon |
Mikrotik | CRS310-1G-5S-4S+IN | Yes**11 | 10G | 1 | 5 | 4 | - | No | Nearly Silent | 8-20w | No | 199$ | Amazon |
Mikrotik | CRS326-4C+20G+2Q+RM | Yes**11 | 40G | 20-24+1 | - | 0-4 | 2 | Yes | 36-54 dBA | 31-70w | No | 999$ | Amazon |
Mikrotik | CRS504-4XQ-IN | Yes**11 | 100G | - | - | - | 48 | Yes | Nearly Silent | 11-41w | No | 650$ | Amazon |
Mikrotik | CRS510-8XS-2XQ-IN | Yes**11 | 100G | 1 | - | 8 | 2 | Yes | Silent - 40 dBA | 15-45w | No | 999$ | Amazon |
Mikrotik | CRS518-16XS-2XQ-RM | Yes**11 | 100G | 1 | - | 1612 | 28 | No | Nearly Silent | 15-95w | No | 1,595$ | Amazon |
Mikrotik | CSS318-16G-2S+IN | No | 10G | 16 | - | 2 | - | No | Fanless | 10-13w | No | 139$ | nan |
Mikrotik | CSS610-8G-2S+IN | No | 10G | 8 | - | 2 | - | No | Fanless | 5-11w | No | 110$ | Amazon |
Unifi | USW-Aggregation | No | 10G | - | - | 8 | - | No | Silent? | 10-30w | No | 269$ | Amazon |
Unifi | USW-Pro-Aggregation | Yes**13 | 25G | - | - | 32 | - | No | Unknown | ?? - 100w | No | 899$ | Amazon |
Mikrotik¶
All of the listed Mikrotik options are acquired new.
Mikrotik hardware will either run RouterOS, or SwOS.
SwOS is a very basic interface, which only exposes basic layer 2 switching, and vlan functionality.
RouterOS, exposes the kitchen sink of networking features.
Warning
Do- check for hardware offload for the features you are interested in, however, as non-hardware offloaded features may have a gimped backplane connection.
For example- My CRS504-4XQ can handle line-speed 100G routing, without breaking a sweat.
If I enable a non-offloaded feature, such as smart-queues, NAT, or non-offloaded filtering- it would need to be process by the CPU. The CPU link works at 1Gbps MAX
This is 100 times slower.
I have found Amazon to be the best place to buy. Mikrotik only sells to other distributors.
Mikrotik CRS518-16XS-2XQ-RM¶
- Price: $1,595
- Capabilities:
- 2x 100G QSFP+, 16x25G SFP+, 1x 10/100M RJ45
- This is a layer 3 switch. If using this for routing, make sure your features are hardware offloaded. The CPU is not very powerful.
- Notes:
- Runs RouterOS Only.
- Links:
Mikrotik CRS510-8XS-2XQ-IN¶
- Price: $999
- Capabilities:
- 2x 100G QSFP+, 8x25G SFP+, 1x 10/100M RJ45
- This is a layer 3 switch. If using this for routing, make sure your features are hardware offloaded. The CPU is not very powerful.
- Can be powered via POE.
- This- is essentially the same hardware as the CRS504-4XQ, but, with two of the 100G QSFP+ ports instead broken out into 8x 25G SFP+
- Notes:
- Runs RouterOS Only.
- Links:
Mikrotik CRS504-4XQ-IN¶
- Price: $650
- Capabilities:
- Ports can support ONE OF:
- 1x 100GBe
- 1x 40GBe
- 4x 25GBe
- 4x 10GBe
- Silent and low power usage (~25W without load).
- With 3x 100G + 4x10G, This switch consumes around 35 watts in my lab.
- Cheapest 100GBe switch option. (One of the cheapest options for 25G too)
- This is a layer 3 switch.
- Can be powered via POE.
- Ports can support ONE OF:
- Notes:
- Use the 1G port for management only; it does not share a backplane.
- This is the central layer 3 router for my lab. Excellent switch.
- Runs RouterOS Only.
- Links:
Mikrotik CRS326-4C+20G+2Q+RM¶
- Price: $999
- Capabilities:
- 20x Multi-gig RJ45 (1/2.5/5G)
- 4x "Combo" ports. Either Multi-gig RJ45, or 10G SFP+
- 1x 10/100m Management Port
- Noisy at startup, but, pretty quiet when running.
- Layer 3** (If you use this as a router, make SURE the features you will be using are hardware offloaded. If the CPU processes packets- performance will be very, very slow.)
- Notes:
- RouterOSv7
- Links:
Mikrotik CRS310-1G-5S-4S+IN¶
- Price: $199
- Capabilities:
- 5x 1G SFP, 4x 10G SFP+, 1x 1G RJ45
- Nearly silent. Low energy
- Layer 3 (Limited to around 1G of routing throughput.)
- Notes:
- SwOS or RouterOSv7
- Links:
Mikrotik CRS309-1G-8S+IN¶
- Price: $269
- Capabilities:
- 8 port SFP+, with 1x 1G RJ45
- Silent and efficient.
- Can be powered via POE
- Notes:
- While this is a layer 3 switch- It does not have dedicated hardware offload, but, can offer reasonable routing performance (not linespeed!)
- Can run either RouterOS, or SwOS
- Links:
Mikrotik CRS305-1G-4S+IN¶
- Price: $120
- Capabilities:
- 4-port 10G Layer 3 switch.
- Silent and efficient.
- Notes:
- While this is a layer 3 switch- It does not have dedicated hardware offload, and is barely capable of routing 1Gbit/s. I only recommend using this for layer 2.
- Can run either RouterOS, or SwOS
- Links:
Mikrotik CRS304-4XG-IN¶
- Price: $199
- Capabilities:
- 4 port layer 3 10G switch with 1x 1G Management Port
- Silent and Efficient.
- Very cost-effective 10G router.
- This is a layer 3 switch.
- Can be powered via POE.
- Multi-gig capabilities (2.5G, 5G)
- Notes:
- Use the 1G port for management only; it does not share a backplane.
- This is the central layer 3 router for my lab. Excellent switch.
- Runs RouterOS Only.
- Links:
Mikrotik CSS610-8G-2S+IN¶
- Price: $139
- Capabilities:
- 16x1G + 2x 10G SFP+
- Silent and efficient.
- Layer 2
- Notes:
- SwOS only.
- I have been personally running one of these for 2 years now. Simple, and effective.
- Links:
Mikrotik CSS318-16G-2S+IN¶
- Price: $139
- Capabilities:
- 16x1G + 2x 10G SFP+
- Silent and efficient.
- Layer 2
- Notes:
- SwOS only.
- Links:
Brocade¶
All of the following brocade switches are acquired used, from eBay or other sources.
If you decide on picking up a brocade switch, I strongly recommend checking out Fohdeesha's Website
These are best acquired from eBay, or other resellers.
Brocade ICX6610¶
- Price: $100 (Can go as low as $40)
- Capabilities:
- Line-speed Layer 3 routing across all ports.
- 2x 40G QSFP, 16x 10G SFP+, 48x 1G POE.
- Supports BGP, OSPF, RIP, NTP, and DHCP server.
- Notes:
- EXTREMELY loud. (Extremely difficult to make these quieter)
- High power consumption (~150W average).
- Absolute unit of a switch. Can get these dirt-cheap. Just- be prepared for it to be hot and loud.
- Comes in 24, or 48 port variants, with, and without POE.
- Links:
Brocade ICX6450¶
- Price: $60 average
- Capabilities:
- 4x 10G SFP + 24 or 48 port options (POE optional).
- Supports OSPF and RIP (no BGP).
- Notes:
- Fan mods are easy and can reduce power usage (~40-50W).
- Runs nearly silent with fan mods.
- Comes in 24, or 48 port variants, with, and without POE.
- Links:
Brocade ICX7150¶
- Price: $100-150 average
- Capabilities:
- 4-8 10G SFP+, 24-48 1G, with optional POE. Depending on configuration.
- Supports OSPF and RIP (no BGP).
- Notes:
- Very quiet. Silent when ran in fanless mode.
- Comes in 24, or 48 port variants, with, and without POE.
- Links:
Brocade ICX7250¶
- Price: $50-150 average
- Capabilities:
- 4-8 10G SFP+, 24-48 1G, with optional POE. Depending on configuration.
- Supports OSPF and RIP (no BGP).
- Notes:
- Pretty quiet under normal working conditions. 40-45dBA on spec sheets.
- Can be ran in fanless mode with reduced POE output.
- Comes in 24, or 48 port variants, with, and without POE.
- 30-50w consumption at idle, without POE. From spec sheets.
- Links:
Mellanox¶
You can pick up used Mellanox infiniband switches for pretty cheap.
Warning
The unmanaged switches are drastically cheaper for a reason.
They REQUIRE an external subnet manager. AFAIK, they are also infiniband-mode only.
For- these reasons, only links to the MANAGED switches are included.
The models ending with an odd-number are unmanaged.
Mellanox SX6036¶
- Price: $75-150 average
- Capabilities:
- 36 QSFP+ (40 / 56GBe)
- Notes:
- This switch supports infiniband.
- This- is a very cost-effective switch, for 40GBe. (or 56G infiniband)
- As I do not own one of these- I cannot give more details.
- Links:
Other Models¶
Unifi¶
Unifi's switches are silent, and efficient.
I generally recommend buying these from Unifi.
Unifi USW-Aggregation¶
- Price: $269
- Capabilities:
- 8x SFP+ (10GBe)
- Notes:
- 8 port layer 2 10G Switch. Silent.
- My switch with all 8 ports in use, with either fiber modules, or DACs, uses around 10w average.
- A solid layer 2 switch.
- Links:
Unifi USW-Pro-Aggregation¶
- Price: $899
- Capabilities:
- 28x 10G SFP+
- 4x 25G SFP+
- NOTE: Does not support mixed speed!! All of these ports must run at the same speed.
- Either- all four are 25G, or all four are 10G.
- Mixed speed is supported for 10G or below.
- This means, you CANNOT run 2x 25G, and 2x 10G. They will instead run at 4x 10G.
- Notes:
- 32 port "layer 3" switch.
- I STRONGLY do not recommend buying a Unifi switch, for its layer 3 capabilities. If you buy this switch, Then consider it a layer 2 switch.
- Links:
Additional Notes¶
Configuration¶
-
Most NICs arrive ready to use. Use
ethtool
to adjust hardware offload parameters if needed: Documentation on ConnectX-3 Configuration -
For setting port modes between ETH/IB: Documentation on ETH/IB Mode
Disclaimers¶
This content is not sponsored
None of the content, or products used in this post was sponsored.
Any products tested, were purchased out of my pocket, and tested without cooperation with the manufacturer.
Amazon Affiliate Links Used
This post DOES include Amazon affiliate links. If you found this content useful, please consider buying the products displayed using the provided links.
You will pay the same amount as normal, however, it does provide a small benefit to me. This benefit is usually used to purchase other products and hardware for which I can review / blog about.
I do not display advertisements on this site. As such, the only compensation from this service, comes from affiliate links. I do not ask for, or even accept donations.
E-Bay Affiliate Links Used
This post DOES include EBay affiliate links. If you found this content useful, please consider buying the products displayed using the provided links.
You will pay the same amount as normal, however, it does provide a small benefit to me. This benefit is usually used to purchase other products and hardware for which I can review / blog about.
I do not display advertisements on this site. As such, the only compensation from this service, comes from affiliate links. I do not ask for, or even accept donations.
-
The Dell R887V is based on the ConnectX-4 CX4121C While the rx3x generation is not listed as compatible- I CAN confirm this does work in my r730XD.
-
These are dirt-cheap for a reason. I strongly recommend buying ConnectX-4 or newer NICs. The price difference is not much, and they are better supported.
-
I generally recommend avoiding RJ45 for 10G. The SFP+ modules use 10GBase-T. These run extremely hot, and have high energy consumption.
-
This is DELL Specific. I can confirm this works on both r720XD, and r730XD. Please do research to ensure if this is compatible with your chassis.
-
This is DELL Specific. Do NOT buy this unless you are absolutely certain this is what you want. These are EXTREMELY picky on modules. I was unable to get my server to boot without a supported module. Neither my Cisco, Mellanox, or Unifi SFP+ modules, or DACs would function.
-
Unmanaged Infiniband Switch.
REQUIRES external subnet manager.
Does not support ethernet mode to my knowledge.
Do not buy these, unless you know EXACTLY what you are purchasing. ↩↩↩
-
56Gb when running in Infiniband Mode.
Supports 1/10/40G in ethernet mode.
Note- IPoIB packets are processed by the CPU of the host typically- Do not recommend using IPoIB. ↩↩↩↩
-
This Mikroktik switch can support various speed configurations.
Each QSFP+ port can do ONE of:
- 1x 100G
- 1x 50G
- 1x 40G
- 4x 25G
- 4x 10G
- 4x 5G
- 4x 2.5G
- 4x 1G
- 4x 100M
-
Two of the rear 40G ports can only be used in 4x10G breakout. ↩↩↩↩↩↩↩↩
-
Mikrotik hardware can perform layer 3 routing without having dedicated hardware offload.
If using the layer 3 functionality, double-check Mikrotik's documentation to ensure your required features are hardware offloaded.
The majority of Mikrotik switches have limited CPU, and are not very effective for doing high-speed routing without hardware offload. ↩↩↩↩↩↩↩↩
-
These SFP+ ports are 25GBe. ↩
-
Don't buy a Unifi switch for its layer 3 capabilities.
You will be disappointed. Treat these as a layer 2 switch, and you will sleep better at night. ↩