POWER8

IBM Power E870 can be configured with up to 80 POWER8 cores and 8 TB of RAM.

POWER8 is a family of superscalar symmetric multiprocessors based on the Power Architecture, and introduced in August 2013 at the Hot Chips conference. The designs are available for licensing under the OpenPOWER Foundation, which is the first time for such availability of IBM's highest-end processors.^[1][2]

Systems based on POWER8 became available from IBM in June 2014.^[3] According to Ken King at IBM, systems and POWER8 processor designs made by other OpenPOWER members will be available in early 2015,^[4] but Tyan seems to be ready to ship earlier than that, in October 2014.^[5]

Design

POWER8 is designed to be a massively multithreaded chip, with each of its cores capable of handling eight hardware threads simultaneously, for a total of 96 threads executed simultaneously on a 12-core chip. The processor makes use of very large amounts of on- and off-chip eDRAM caches, and on-chip memory controllers enable very high bandwidth to memory and system I/O. For most workloads, the chip is said to perform two to three times as fast as its predecessor, the POWER7.^[6]

POWER8 comes in 4-, 6-, 8-, 10- and 12-core variants;^[7][8] each version is fabricated in a 22 nm silicon on insulator (SOI) process using 15 metal layers. The 12-core version consists of 4.2 billion transistors^[9] and is 650 mm² large while the 6-core version is only 362 mm² large.^[3]

CAPI

Where previous POWER processors use the GX++ bus for external communication, POWER8 removes this from the design and replaces it with the CAPI port (Coherent Accelerator Processor Interface) that is layered on top of PCI Express 3.0. The CAPI port is used to connect auxiliary specialized processors such as GPUs, ASICs and FPGAs.^[10][11] Units attached to the CAPI bus can use the same memory address space as the CPU, thereby reducing the computing path length. At the 2013 ACM/IEEE Supercomputing Conference, IBM and Nvidia announced an engineering partnership to closely couple POWER8 with Nvidia GPUs in future HPC systems,^[12] with the first of them announced as the Power Systems S824L.

On October 14th, 2016, IBM announced the formation of OpenCAPI, a new organization to spread adoption of CAPI to other platforms. Initial members are Google, AMD, Xilinx, Micron and Mellanox.^[13]

OCC

POWER8 also contains a so-called on-chip controller (OCC), which is a power and thermal management microcontroller based on a PowerPC 405 processor. It has two general-purpose offload engines (GPEs) and 512 KB of embedded static RAM (SRAM), together with the possibility to access the main memory directly, while running an open-source firmware. OCC manages POWER8's operating frequency, voltage, memory bandwidth, and thermal control for both the processor and memory; it can regulate voltages through 1,764 integrated voltage regulators (IVRs) on the fly. Also, the OCC can be programmed to overclock the POWER8 processor, or to lower its power consumption by reducing the operating frequency (which is similar to the configurable TDP found in some of the Intel and AMD processors).^{[14][15][16][17]}

Centaur

The memory controllers on the POWER8 chips are specified to use either DDR3 or DDR4 memory but are designed to be future-proof by being generic memory controllers paired to an external component called Centaur that acts as a memory buffer, L4 cache chip and the actual memory controllers. The current Centaur chip is using DDR3 memory, but a future version can use DDR4 or some other memory technology without the need to modify the design of the POWER8 chip itself.

Every Centaur is connected to the POWER8 by means of three high-speed links (2:1 Read:Write), each running at 9.6 GB/s, for a total bandwidth of 28.8 GB/s, with a 40 ns latency. Centaur contains 16 MB of eDRAM which can be used as L4 cache by the processor. Each POWER8 can be linked to up to eight Centaur chips allowing for up to 1 TB of memory per socket, with an aggregated 128 MB L4 cache and 230 GB/s sustained memory bandwidth in and out of the processor, with a total of 32 DRAM ports and 410 GB/s peak memory bandwidth at the DRAM. Centaur chips are mounted onto DRAM DIMM modules.^[1][18][19]

The Centaur chips are fabricated with a process similar to that of the POWER8.

Specifications

The POWER8^[20][21] core has 64 KB L1 data and 32 KB L1 instruction caches. Each core can issue ten instructions and dispatch eight during each cycle to 16 execution pipes: two fixed-point pipelines, two load/store pipelines, two load pipelines, four double-precision floating-point pipelines that can also act as eight single-precision floating-point pipelines, two VMX pipelines, one cryptographic pipeline, one decimal floating-point pipeline, one condition register pipeline, and one branch execution pipeline.^[22]

It has a larger issue queue with 4×16 entries, improved branch predictors and can handle twice as many cache misses. Each core is eight-way hardware multithreaded and can be dynamically and automatically partitioned to have either one, two, four or all eight threads active.^[1] POWER8 also added support for hardware transactional memory.^[23][24][25] IBM estimates that each core is 1.6 times as fast as the POWER7 in single-threaded operations.

A POWER8 processor is a 12-chiplet design with variants consisting of either 4, 6, 8, 10 or 12 chiplets, in which one chiplet consists of one core, 512 KB of SRAM L2 cache on a 64-byte wide bus (which is twice as wide as on its predecessor^[1]), and 8 MB of L3 eDRAM cache per chiplet shareable among all chiplets.^[7] Thus, a six-chiplet processor would have 48 MB of L3 eDRAM cache, while a 12-chiplet processor would have a total of 96 MB of L3 eDRAM cache. The chip can also utilize an up to 128 MB of off-chip eDRAM L4 cache using Centaur companion chips. The on-chip memory controllers can handle 1 TB of RAM and 230 GB/s sustained memory bandwidth. The on-board PCI Express controllers can handle 48 GB/s of I/O to other parts of the system. The cores are designed to operate at clock rates between 2.5 and 5 GHz.^[17]

The six-core version is mounted in pairs on dual-chip modules (DCM) in IBM's Scale Out servers. In most configurations not all cores are active, resulting in a variety of configurations where the actual core count differs. The 12-core version has not been released in any system as of May 2014.

IBM's single-chip POWER8 module is called Turismo^[5] and the dual-chip variant is called Murano.^[26] PowerCore's modified version is called CP1.

POWER8 with NVLink

This is a revised version of the original 12-core POWER8 from IBM, and used to be called POWER8+. The main new feature is that it has support for Nvidia's bus technology NVLink, connecting up to four NVLink devices directly to the chip. IBM removed the A Bus and PCI interfaces for SMP connections to other POWER8 sockets and replaced them with NVLink interfaces. Connection to a second CPU socket are now provided via the X Bus. Besides that and a slight size increase to 659 mm², the differences seem minimal compared to previous POWER8 processors.^{[27][28][29][30]}

Licensees

On 19 January 2014, the Suzhou PowerCore Technology Company announced that they will join the OpenPOWER Foundation and license the POWER8 core to design custom-made processors for use in big data and cloud computing applications.^[31][32]

Variants

• IBM Murano – a 12-core processor with two six-core chips. Scale-out processor is available in configurations with disabled cores.
• IBM Turismo – a single-chip 12-core processor. Scale-up processor is commercially available for licensing and purchase in configurations with disabled cores.
• PowerCore CP1 – a POWER8 variant with revised security features due to export restrictions between United States and China that will be manufactured in Global Foundries (formerly IBM's plant) factory in East Fishkill, New York. To be released in 2015.^[33][34]

Systems

Rear view of an E870, in which the system control unit is on top and the system node is in the middle.^[35]

IBM

Scale Out servers, supporting one or two sockets each carrying a dual-chip module with two six-core POWER8 processors. They come in either 2U or 4U form factors, and one tower configuration. The "L" versions run only Linux, while the others run AIX, IBM i and Linux. The "LC" versions are built by OpenPOWER partners.^[36][37][38]

• Power Systems S812L – 1× POWER8 DCM (4, 6 or 8 cores), 2U
• Power Systems S814 – 1× POWER8 DCM (6 or 8 cores), 4U or tower
• Power Systems S822 and S822L – 1× or 2× POWER8 DCM (6, 10, 12 or 20 cores), 2U
• Power Systems S824 and S824L – 1× or 2× POWER8 DCM (6, 8, 12, 16 or 24 cores), 4U
• Power Systems S821LC "Stratton" – 2× POWER8 SCM (8 or 10 cores), 1U. Up to 512 GB DDR4 RAM buffered by four Centaur L4 chips. Manufactured by Supermicro.^[39]
• Power Systems S822LC for Big Data "Briggs" – 2× POWER8 SCM (8 or 10 cores), 2U. Up to 512 GB DDR4 RAM buffered by four Centaur L4 chips. Manufactured by Supermicro.^[39]

Enterprise servers, supporting nodes with four sockets, each carrying 8-, 10- or 12-core modules, for a maximum of 16 sockets, 128 cores and 16 TB of RAM. These machines can run AIX, IBM i, or Linux.^[40]

• Power Systems E850 – 2×, 3× or 4× POWER8 DCM (8, 10 or 12 cores), 4U
• Power Systems E870 – 1× or 2× 5U nodes, each with four sockets with 8- or 10-core POWER8 single-chip modules, for up to a total of 80 cores
• Power Systems E880 – 1x, 2x, 3x or 4x 5U nodes, each with four sockets with 8- or 12-core POWER8 single-chip modules for up to a total of 192 cores

High performance computing:

• Power Systems S812LC – 1× POWER8 SCM (8 or 10 cores), 2U. Manufactured by Tyan.^[41]
• Power Systems S822LC "Firestone" – 2× POWER8 SCM (8 or 10 cores), 2U. Two Nvidia Tesla K80 GPUs and up to 1 TB commodity DDR3 RAM. Manufactured by Wistron.^{[33][41][42][43]}
• Power Systems S822LC for HPC "Minsky" – 2× POWER8+ SCM (8 or 10 cores), 2U. Up to four NVLinked Nvidia Tesla P100 GPUs and up to 1 TB commodity DDR4 RAM. Manufactured by Wistron.^[39][44]

Tyan

• An ATX motherboard with one single-chip POWER8 socket called the SP010GM2NR.^[5]
• Palmetto GN70-BP010, OpenPower reference system. 2U server, with one four-core POWER8 SCM, four RAM sockets, based on a Tyan's motherboard.^[5][45]
• Habanero TN-71-BP012. 2U, with one 8 core POWER8 SCM, 32 RAM sockets^[33][43][45]
• GT75-BP012. 1U, with a single 8- or 10-core POWER8 SCM and 32 sockets for RAM modules^[46]

Google

Google has shown a motherboard with two sockets, intended for internal use only.^[47][48]

StackVelocity

StackVelocity has designed a high-performance reference platform, Saba.

Inspur

Inspur has made a deal with IBM to develop server hardware based on POWER8 and related technologies.^[49][50]

• 4U server, two POWER8 sockets.^[51]

Cirrascale

RM4950 – 4U, 4-core POWER8 SCM with four Nvidia Tesla K40 accelerators. Based on Tyan's motherboard.^{[33][42][43][52]}

Zoom Netcom

RedPOWER C210 and C220 – 2U and 4U servers with two POWER8 sockets and 64 sockets for RAM modules.^[33][53]

RedPOWER C310 and C320 – 2U and 4U servers with two CP1 sockets.^[53]

ChuangHe

OP-1X – 1U, single socket, 32 RAM slots.^[33][54]

Rackspace

Barreleye – 1U, 2 socket, 32 RAM slots. Based on the Open Compute Project platform for use in their OnMetal service.^{[43][54][55][56][57]}

See also

• IBM POWER microprocessors
• OpenPOWER Foundation
• POWER7
• POWER9
• Power Architecture
• PowerPC A2

References

External links

Wikimedia Commons has media related to POWER8 systems.

• POWER8 Overview, IBM Power Systems (PDF)