NVIDIA Mellanox MCS7500 / CS7500 648-Port EDR 100Gb/s InfiniBand Director Switch. MCS7500 is the NVIDIA ordering part number for the CS7500 modular director-class InfiniBand chassis: a 28U core switch carrying up to 648 EDR 100Gb/s QSFP28 ports, 130Tb/s aggregate bi-directional switching capacity and 400ns port-to-port latency, built from hot-swappable Switch-IB and Switch-IB 2 leaf and spine blades, x86 management modules, 10 power supplies and 20 fan units. It is a chassis director switch for the fabric core - not a fixed 1U top-of-rack switch, and not an Ethernet platform. Refurbished, supplied by T.E.S IT-SOLUTIONS.
Engineering Context
EDR runs four lanes per QSFP28 port at 25.78125Gb/s with 64/66b encoding for 100Gb/s per port, with FEC per IEEE Std 802.3ap-2007 clause 74. The chassis is IBTA 1.2.1 and 1.3 compliant, with 9 virtual lanes (8 data + 1 management), 256-byte to 4Kbyte MTU and a 4 x 48K-entry linear forwarding database. Internally it is a two-stage non-blocking fat-tree: 36-port leaf blades carry the external cabling, 36-port spine blades form the internal crossbar, and a full chassis holds 18 leaf, 18 spine and 2 management modules. CS7500 was the first director-class switch with SHARP in-network computing, running MPI and SHMEM collective reductions inside Switch-IB 2 silicon. Fully populated it draws 6,367.1W typical (ATIS) and 13,531W maximum, and weighs 369.5kg - plan floor loading and PDU capacity in advance.
Deployment and Use Cases
Core and spine layer of large fat-tree HPC fabrics aggregating hundreds of EDR endpoints without oversubscription. Backend storage fabric for Lustre, IBM Spectrum Scale (GPFS), BeeGFS and Ceph over InfiniBand. Central interconnect for GPU pods standardised on EDR, including DGX-1 and DGX-2 generation estates. MPI workloads (HPC-X, Open MPI, MPICH) in CFD, weather modelling, molecular dynamics, seismic processing and genomics. Fabric consolidation replacing several 36-port or 216/324-port EDR switches with one managed chassis under a single subnet manager.
Technical Specifications
Brand / OPN: NVIDIA Mellanox, MCS7500 (platform family CS7500)
Type: Modular director-class InfiniBand chassis switch, 28U + 1U shelf
Silicon: Mellanox Switch-IB and Switch-IB 2
Ports: up to 648 x EDR 100Gb/s, QSFP28
Per-port rate: 100Gb/s (4 x 25.78125Gb/s NRZ lanes, 64/66b)
Aggregate capacity: 130Tb/s bi-directional
Port-to-port latency: 400ns
Protocol: InfiniBand EDR only; backward compatible to FDR and QDR
Blades (full): 18 leaf + 18 spine + 2 management, 36 ports each
Blade OPNs: MSB7560-E / MSB7570-E (SHARP), MSB7510-E2 / MSB7520-E2 (no SHARP), MMB7500 management
Fabric management: on-board Subnet Manager up to 2,000 nodes; MLNX-OS; UFM agent
Power supplies: 10 x MTDF-PS-A 2,500W hot-swap, 80 PLUS Gold+
PSU redundancy: combined (min. 5), N+1 (6), N+N grid-redundant (1-5 grid A, 6-10 grid B)
AC input: 180-265VAC, 47-63Hz; 220VAC only, no 110VAC; min. 12A per source
Cooling: 20 hot-swap fans (MTDF-FAN-A leaf side, MTDF-FAN-B spine side), front-to-rear
Condition: Refurbished - tested by T.E.S IT-SOLUTIONS
Compatibility and Parent Systems
The CS7500 sits at the core of an EDR-generation InfiniBand fabric. Typical endpoints are ConnectX-4 and ConnectX-4 VPI (MCX455A / MCX456A), ConnectX-5 and ConnectX-5 Ex (MCX555A / MCX556A) in EDR InfiniBand mode, and ConnectX-6 negotiated down to EDR. Cabling is QSFP28: MCP1600-E passive DACs, MFA1A00-E AOCs, MMA1B00-E100 multimode SR4 and MMA1L30-CM single-mode CWDM4 modules. Typical leaf partners of the same generation are the 36-port 1U SB7800 (managed) and SB7890 (externally managed). Predecessor: SX6536, the 648-port FDR 56Gb/s director. Successor: CS8500, the HDR 200Gb/s director. Subnet Manager and UFM entitlements are not transferred with refurbished hardware.
What This Is NOT
NOT an Ethernet switch. InfiniBand only; the Ethernet family of this era is Spectrum SN2000.
NOT HDR or NDR. Maximum per-port rate is EDR 100Gb/s. HDR 200Gb/s needs Quantum (QM8700 / CS8500), NDR 400Gb/s needs Quantum-2. No field upgrade path.
NOT a fixed 1U or 2U switch. 28U modular chassis plus a 1U installation shelf.
NOT OSFP or QSFP-DD. All external ports are QSFP28; NDR OSFP optics do not fit.
NOT a 110VAC device. 220VAC operation only, at least 12A per AC source.
NOT sold with software entitlements and NOT supplied with QSFP28 transceivers, DAC or AOC cabling unless listed on the quotation.
Frequently Asked Questions
Q. What is the difference between MCS7500 and CS7500?
They are the same machine. CS7500 is the platform name; MCS7500 is the NVIDIA ordering part number, described by NVIDIA as a 130Tb/s, 648-port EDR chassis switch including 20 fans and 10 power supplies in an N+N configuration.
Q. How many ports and how much capacity does the MCS7500 provide?
Up to 648 EDR 100Gb/s QSFP28 ports in 28U, 130Tb/s aggregate bi-directional capacity and 400ns port-to-port latency. Full population is 18 leaf and 18 spine blades at 36 ports each, plus 2 management modules.
Q. Does the MCS7500 run its own subnet manager, or do I need UFM?
The MMB7500 management module runs MLNX-OS on an x86 dual-core CPU with an on-board Subnet Manager rated for fabrics of up to 2,000 nodes, so no external SM appliance is mandatory at that scale. Larger or multi-subnet fabrics use NVIDIA UFM. Entitlements are not included with refurbished hardware.
Q. Does the MCS7500 support SHARP in-network computing?
SHARP is a Switch-IB 2 capability and depends on the blades fitted. MSB7560-E leaf and MSB7570-E spine blades support SHARP; NVIDIA explicitly lists MSB7510-E2 and MSB7520-E2 as having no SHARP support. Confirm blade part numbers on the chassis being quoted.
Q. What power and cooling does the MCS7500 need?
Ten MTDF-PS-A 2,500W hot-swap supplies, 220VAC only (180-265VAC, 47-63Hz), at least 12A per source. Five PSUs are the minimum for full capacity, six for N+1, ten across two grids for N+N. Typical ATIS draw 6,367.1W, maximum 13,531W. Cooling is 20 hot-swap fans, front-to-rear.
Q. Which adapters and cables pair with the MCS7500?
ConnectX-4, ConnectX-4 VPI, ConnectX-5 and ConnectX-5 Ex in InfiniBand mode, and ConnectX-6 negotiated to EDR. Cabling is QSFP28: MCP1600-E DACs, MFA1A00-E AOCs, MMA1B00-E100 or MMA1L30-CM optics. FDR and QDR equipment interoperates at the lower rate.
Why T.E.S IT-SOLUTIONS
T.E.S IT-SOLUTIONS (tes-itsolutions.com) supplies refurbished NVIDIA Mellanox InfiniBand hardware to B2B, HPC and research customers across the EU and worldwide. Director-class systems are bench-verified before shipment: chassis power-up, management module response, blade seating and link training where test topology allows, PSU and fan redundancy check and cosmetic inspection. EU stock, 1-3 business day handling and technical consultation on HPC fabric design at no extra cost.