HyperTransport

Applications

Front-side bus replacement

The primary use for HyperTransport is to replace the Intel-defined front-side bus, which is different for every type of Intel processor. For instance, a Pentium cannot be plugged into a PCI Express bus directly, but must first go through an adapter to expand the system. The proprietary front-side bus must connect through adapters for the various standard buses, like AGP or PCI Express. These are typically included in the respective controller functions, namely the northbridge and southbridge.

In contrast, HyperTransport is an open specification, published by a multi-company consortium. A single HyperTransport adapter chip will work with a wide spectrum of HyperTransport enabled microprocessors.

AMD used HyperTransport to replace the front-side bus in their Opteron, Athlon 64, Athlon II, Sempron 64, Turion 64, Phenom, Phenom II and FX families of microprocessors. And AMD used HyperTransport to replace the back-side bus in their Athlon 64 X2, Athlon II, Phenom, Phenom II and FX families of microprocessors.

Multiprocessor interconnect

Another use for HyperTransport is as an interconnect for NUMA multiprocessor computers. AMD used HyperTransport with a proprietary cache coherency extension as part of their Direct Connect Architecture in their Opteron and Athlon 64 FX (Dual Socket Direct Connect (DSDC) Architecture) line of processors. Infinity Fabric used with the EPYC server CPUs is a superset of HyperTransport. The HORUS interconnect from Newisys extends this concept to larger clusters. The Aqua device from 3Leaf Systems virtualizes and interconnects CPUs, memory, and I/O.

Router or switch bus replacement

HyperTransport can also be used as a bus in routers and switches. Routers and switches have multiple network interfaces, and must forward data between these ports as fast as possible. For example, a four-port, 1000 Mbit/s Ethernet router needs a maximum 8000 Mbit/s of internal bandwidth (1000 Mbit/s × 4 ports × 2 directions)—HyperTransport greatly exceeds the bandwidth this application requires. However a 4 + 1 port 10 Gb router would require 100 Gbit/s of internal bandwidth. Add to that 802.11ac 8 antennas and the WiGig 60 GHz standard (802.11ad) and HyperTransport becomes more feasible (with anywhere between 20 and 24 lanes used for the needed bandwidth).

Co-processor interconnect

The issue of latency and bandwidth between CPUs and co-processors has usually been the major stumbling block to their practical implementation. Co-processors such as FPGAs have appeared that can access the HyperTransport bus and become integrated on the motherboard. Current generation FPGAs from both main manufacturers (Altera and Xilinx) directly support the HyperTransport interface, and have IP Cores available. Companies such as XtremeData, Inc. and DRC take these FPGAs (Xilinx in DRC's case) and create a module that allows FPGAs to plug directly into the Opteron socket.

AMD started an initiative named Torrenza on September 21, 2006, to further promote the usage of HyperTransport for plug-in cards and coprocessors. This initiative opened their "Socket F" to plug-in boards such as those from XtremeData and DRC.

Add-on card connector (HTX and HTX3)

A connector specification that allows a slot-based peripheral to have direct connection to a microprocessor using a HyperTransport interface was released by the HyperTransport Consortium. It is known as HyperTransport eXpansion (HTX). Using a reversed instance of the same mechanical connector as a 16-lane PCI Express slot (plus an x1 connector for power pins), HTX allows development of plug-in cards that support direct access to a CPU and DMA to the system RAM. The initial card for this slot was the QLogic InfiniPath InfiniBand HCA. IBM and HP, among others, have released HTX compliant systems.

The original HTX standard is limited to 16 bits and 800 MHz.

In August 2008, the HyperTransport Consortium released HTX3, which extends the clock rate of HTX to 2.6 GHz (5.2 GT/s, 10.7 GTi, 5.2 real GHz data rate, 3 MT/s edit rate) and retains backwards compatibility.

Testing

The "DUT" test connector is defined to enable standardized functional test system interconnection.

Implementations

• AMD AMD64 and Direct Connect Architecture-based CPUs
• AMD chipsets
  • AMD-8000 series
  • AMD 480 series
  • AMD 580 series
  • AMD 690 series
  • AMD 700 series
  • AMD 800 series
  • AMD 900 series
• ATI chipsets
  • ATI Radeon Xpress 200 for AMD processor
  • ATI Radeon Xpress 3200 for AMD processor
• Broadcom (then ServerWorks) HyperTransport SystemI/O controllers
  • HT-2000
  • HT-2100
• Cisco QuantumFlow Processors
• ht_tunnel from OpenCores project (MPL licence)
• IBM CPC925 and CPC945 (PowerPC 970 northbridges) chipsets
• Loongson-3 MIPS processor
• Nvidia nForce chipsets
  • nForce and nForce2 series (link between North and South Bridges)
  • nForce Professional MCPs (Media and Communication Processor)
  • nForce 3 series
  • nForce 4 series
  • nForce 500 series
  • nForce 600 series
  • nForce 700 series
  • nForce 900 series
• PMC-Sierra RM9000X2 MIPS CPU
• Power Mac G5
• Raza Thread Processors
• SiByte MIPS CPUs from Broadcom
• Transmeta TM8000 Efficeon CPUs
• VIA chipsets K8 series

Frequency specifications

* AMD Athlon 64, Athlon 64 FX, Athlon 64 X2, Athlon X2, Athlon II, Phenom, Phenom II, Sempron, Turion series and later use one 16-bit HyperTransport link. AMD Athlon 64 FX (1207), Opteron use up to three 16-bit HyperTransport links. Common clock rates for these processor links are 800 MHz to 1 GHz (older single and multi socket systems on 754/939/940 links) and 1.6 GHz to 2.0 GHz (newer single socket systems on AM2+/AM3 links—most newer CPUs using 2.0 GHz). While HyperTransport itself is capable of 32-bit width links, that width is not currently utilized by any AMD processors. Some chipsets though do not even utilize the 16-bit width used by the processors. Those include the Nvidia nForce3 150, nForce3 Pro 150, and the ULi M1689—which use a 16-bit HyperTransport downstream link but limit the HyperTransport upstream link to 8 bits.

Name

There has been some marketing confusion between the use of HT referring to HyperTransport and the later use of HT to refer to Intel's Hyper-Threading feature on some Pentium 4-based and the newer Nehalem and Westmere-based Intel Core microprocessors. Hyper-Threading is officially known as Hyper-Threading Technology (HTT) or HT Technology. Because of this potential for confusion, the HyperTransport Consortium always uses the written-out form: "HyperTransport."

Infinity Fabric

Infinity Fabric (IF) is a superset of HyperTransport announced by AMD in 2016 as an interconnect for its GPUs and CPUs. When used internally it is called a Global Memory Interconnect (GMI). It is also usable as interchip interconnect for communication between CPUs and CPUs, GPUs and GPUs, or CPUs and GPUs (for Heterogeneous System Architecture), an arrangement known as Infinity Architecture, with the links known as External Global Memory Interconnect (xGMI). The company said the Infinity Fabric would scale from 30 GB/s to 512 GB/s, and be used in the Zen-based CPUs and Vega GPUs which were subsequently released in 2017.

On Zen and Zen+ CPUs, the "SDF" data interconnects are run at the same frequency as the DRAM memory clock (MEMCLK), a decision made to remove the latency caused by different clock speeds. As a result, using a faster RAM module makes the entire bus faster. The links are 32-bit wide, as in HT, but 8 transfers are done per cycle (128-bit packets) compared to the original 2. Electrical changes are made for higher power efficiency. On Zen 2 and Zen 3 CPUs, the IF bus is on a separate clock (FCLK) and so is the unified memory controller (UCLK). The UCLK is either in a 1:1 or a 2:1 ratio to the DRAM clock (MCLK). This avoids a limitation on desktop platforms where maximum DRAM speeds were in practice limited by the IF speed. The bus width has also been doubled. A latency penalty is present when the FCLK is not synchonized with the UCLK. On Zen 4 and later CPUs, the IF bus is able to run at an asynchronous clock to the DRAM, to allow the higher clock speeds that DDR5 is capable of.

Infinity Fabric Link

Professional/Workstation models of AMD GPUs include an Infinity Fabric Link edge connector for connecting the Infinity Fabric buses of GPUs together, bypassing the host PCIe bus. The Link "Bridge" device itself is a printed circuit board with 2 or 4 matching slots. Each GPU family uses a different connector and the Bridge/Link generally only works between the GPUs of the same model. It is therefore similar to the plug-in board version of NVLink.

Example Infinity Architecture

Epyc CPUs based on Zen 5 have internal Infinity Fabric connections of 36 GB/s per core. Each IO die has external Infinity Fabric connectivity on its multifunctional PCIe 5.0/Infinity Fabric serializer/deserializers (SerDes), reusing the PCIe physical layer. It is used for interprocessor communication in two-socket systems, providing 3 or 4 links of 64 GB/s each.

Each Instinct MI250 has four lanes of Infinity Fabric Link of 50 GB/s each for mesh interconnection running the xGMI protocol. It connects to the host through PCIe Gen 4 x16 or Infinity Fabric on top of PCIe PHY. The bandwidth from multiple links, passing through different intermediate GPUs, can be aggregated. For actually achievable performance figures, see Schieffer et al. (2024).

Third-party support

UALink utilizes Infinity Fabric/xGMI as one of its shared memory protocols.

Broadcom produces PCIe switches and network interface cards with xGMI support.