Samsung Sets 'Three-Phase HBM Roadmap': Moving Toward True 3D ZHBM, Stacking DRAM Directly on Computing Chips

Recently, at the Hot Chips technical conference, Sangwook Han from Samsung's DRAM design team gave a presentation, officially unveiling Samsung's three-phase roadmap for HBM evolution. The final point of this roadmap is a brand-new architecture called zHBM—where DRAM is stacked vertically directly above GPU, TPU, and other computing chips, completely eliminating the need for a 2.5D interposer.

According to wccftech's report on August 23, Samsung stated that compared to standard HBM4E, the zHBM solution claims to achieve a 70% reduction in power consumption, a 230% increase in DRAM bandwidth, as well as saving 100W of power per DRAM module, while freeing up 8.3% additional power budget for GPUs.

What is HBM and Where Are the Bottlenecks?

HBM, or High Bandwidth Memory, is currently one of the most critical storage components in AI training and inference systems. It consists of two types of chips:

• C-die (Core Die): Contains DRAM memory cells and allows for vertical stacking of up to 16 layers

• B-die (Base Die): Located at the bottom of the stack, responsible for handling various DRAM control functions, and communicates with GPU and other logic chips via PHY (Physical Interface Layer)

The two are connected via TSVs (Through-Silicon Vias)—vertical conductive channels that penetrate the chip.

According to Wccftech, the current HBM4 offers bandwidth exceeding 3TB/s per stack, with HBM4E pushing further into the 4TB/s range, and HBM5 doubling the bandwidth of HBM4, surpassing 60GB in capacity.

However, continuous bandwidth expansion is facing two major hard constraints: the physical limits of TSV numbers and spacing, and the I/O quantity and speed limitations of the Base Die's PHY interface. Meanwhile, the generational gap in fabrication processes between B-die and computing chips (xPU SoC) continues to narrow—posing both a challenge and an opportunity, which Samsung’s roadmap aims to address.

Phase One: Freeing Up Space for Compute Chips

The central goal of the first phase of Samsung's roadmap is to “reclaim” silicon area from xPU (computing chips).

Samsung has applied D1c and 4nm logic processes to the HBM4 Base Die (B-die), primarily to reduce power consumption and minimize effective area. This marks the true beginning of integrating DRAM with advanced logic processes.

Specific measures include:

• Replacing traditional HBM PHY with D2D interfaces, shortening channel lengths to directly improve energy efficiency, while freeing up valuable silicon area for the XPU

• Offloading the memory controller from the XPU to the B-die of cHBM, which is expected to free 5% to 10% of XPU area, corresponding to a 10% to 20% performance boost

• Introducing SRAM-based Fine-Granularity Cell Repair schemes (Near-MC SRAM-Based Cell Repair), leveraging unused space on the B-die to deploy SRAM repair resources

The side effect of area reduction is the hotspot issue. For this, Samsung has launched Heat Path Block (HPB) technology, built on the cHBM4 solution, which can reduce peak temperatures by more than 35%, covering 50% of the PHY area.

Phase Two: B-die Starts to Gain Compute Capability

The focus of the second phase shifts from “freeing up space” to “adding features”—Samsung defines this as the feature expansion phase.

Expanding memory capacity. As AI large models’ context windows have increased sharply, the demand for KV Cache (Key-Value Cache, memory area for storing intermediate states during model inference) has grown exponentially. Samsung plans to integrate memory extension controllers and PHY into idle silicon area on the Base Die, enabling external connection to LPDDR or HBM and expanding total system memory.

Integrated Processing Elements (PE). Samsung also suggests integrating compute processing units into the Base Die, offloading part of the operations originally performed on the xPU to the memory side, thus reducing D2D bandwidth requirements, along with power and thermal load. This approach is called AHBM (Advanced HBM).

Enhanced reliability and testability. The second phase also involves integrating advanced RAS (Reliability, Availability, Serviceability) sensors, real-time telemetry functions, and on-chip self-test (ATIP) capabilities into the Base Die, improving yield and test coverage.

Phase Three: zHBM—Eliminating the Interposer, DRAM Directly Stacked on Compute Chips

Phase three is the ultimate form of the entire roadmap: zHBM.

Currently, mainstream AI systems use 2.5D packaging architecture, with GPU and HBM placed side by side on the same interposer, and data being transferred laterally via interconnects. zHBM’s concept is to “turn this structure vertical”—stacking DRAM directly above the xPU chip, creating true 3D vertical integration.

Samsung describes the key features of zHBM as follows:

• Distributed I/O: Minimizes data transmission distance within the HBM stack

• 3D structure: Eliminates traditional 2D interfaces, greatly enhancing system efficiency

• I/O power target of about 0.5 pJ/bit: Achieved by removing redundant modules such as SerDes

• Bandwidth boost over 2.3 times, system thermal headroom up to 100W

Samsung’s demonstration involves stacking four zHBM stacks on top of a single XPU.

To realize these goals, Samsung is developing two key packaging technologies: WoW (Wafer on Wafer) and HCB (Hybrid Cube Bonding), to achieve ultra-high I/O density and eventually create a unified SoC-DRAM co-design ecosystem.

The Core Logic Behind the Roadmap

The core narrative of Samsung’s presentation is to reposition the HBM Base Die from a mere “passive data relay” to an “intelligent partner with active computing capability.”

The logic of the three-phase evolution is clear: first, use process upgrades to shrink area and free xPU space (Phase One); next, integrate more features using the gained area, expanding capacity and compute capability (Phase Two); and ultimately, reconstruct the system architecture through 3D vertical integration, making simultaneous breakthroughs in power, bandwidth, and thermal management (Phase Three).

Samsung concluded: "By mastering advanced packaging and unified SoC-DRAM co-design, we will break through the power, area, and capacity bottlenecks of AI systems and pave the way for higher efficiency, performance, and scalability for years to come."