Hubbry Logo
High Bandwidth MemoryHigh Bandwidth MemoryMain
Open search
High Bandwidth Memory
Community hub
High Bandwidth Memory
logo
7 pages, 0 posts
0 subscribers
Be the first to start a discussion here.
Be the first to start a discussion here.
Contribute something
High Bandwidth Memory
High Bandwidth Memory
High Bandwidth Memory
View on Wikipedia
from Wikipedia
Computer memory and data storage types
General
Volatile
RAM
Historical
Non-volatile
ROM
NVRAM
Early-stage NVRAM
Analog recording
Optical
In development
Historical

High Bandwidth Memory (HBM) is a computer memory interface for 3D-stacked synchronous dynamic random-access memory (SDRAM) initially from Samsung, AMD and SK Hynix. It is used in conjunction with high-performance graphics accelerators, network devices, high-performance datacenter AI ASICs, as on-package cache in CPUs[1] and on-package RAM in upcoming CPUs, and FPGAs and in some supercomputers (such as the NEC SX-Aurora TSUBASA and Fujitsu A64FX).[2] The first HBM memory chip was produced by SK Hynix in 2013,[3] and the first devices to use HBM were the AMD Fiji GPUs in 2015.[4][5]

HBM was adopted by JEDEC as an industry standard in October 2013.[6] The second generation, HBM2, was accepted by JEDEC in January 2016.[7] JEDEC officially announced the HBM3 standard on January 27, 2022,[8] and the HBM4 standard in April 2025.[9][10]

Technology

[edit]
Type Release max data
rate speed
per pin		 Stack per Stack
max capacity max data rate
HBM 1 Oct 2013 1.0 Gb/s 8×128 bit 4 dies × 1 GB = 4 GB 128 GB/s
HBM 2 Jan 2016 2.4 Gb/s 8 dies × 1 GB = 8 GB 307 GB/s
HBM 2E Aug 2019 3.6 Gb/s 12 dies × 2 GB = 24 GB 461 GB/s
HBM 3 Jan 2022 6.4 Gb/s 16×64 bit 12 dies × 2 GB = 24 GB 819 GB/s
HBM 3E May 2023 9.8 Gb/s 16 dies × 3 GB = 48 GB 1229 GB/s
HBM 4 April 2025 8 Gb/s 32×64 bit 16 dies × 4 GB = 64 GB 2048 GB/s

HBM achieves higher bandwidth than DDR4 or GDDR5 while using less power, and in a substantially smaller form factor.[11] This is achieved by stacking up to eight DRAM dies and an optional base die which can include buffer circuitry and test logic.[12] The stack is often connected to the memory controller on a GPU or CPU through a substrate, such as a silicon interposer.[13][14] Alternatively, the memory die could be stacked directly on the CPU or GPU chip. Within the stack the dies are vertically interconnected by through-silicon vias (TSVs) and microbumps. The HBM technology is similar in principle but incompatible with the Hybrid Memory Cube (HMC) interface developed by Micron Technology.[15]

HBM memory bus is very wide in comparison to other DRAM memories such as DDR4 or GDDR5. An HBM stack of four DRAM dies (4‑Hi) has two 128‑bit channels per die for a total of 8 channels and a width of 1024 bits in total. A graphics card/GPU with four 4‑Hi HBM stacks would therefore have a memory bus with a width of 4096 bits. In comparison, the bus width of GDDR memories is 32 bits, with 16 channels for a graphics card with a 512‑bit memory interface.[16] HBM supports up to 4 GB per package.

The larger number of connections to the memory, relative to DDR4 or GDDR5, required a new method of connecting the HBM memory to the GPU (or other processor).[17] AMD and Nvidia have both used purpose-built silicon chips, called interposers, to connect the memory and GPU. This interposer has the added advantage of requiring the memory and processor to be physically close, decreasing memory paths. However, as semiconductor device fabrication is significantly more expensive than printed circuit board manufacture, this adds cost to the final product.

  • HBM DRAM die
    HBM DRAM die
  • HBM controller die
    HBM controller die
  • HBM memory on an AMD Radeon R9 Nano graphics card's GPU package
    HBM memory on an AMD Radeon R9 Nano graphics card's GPU package

Interface

[edit]
Cut through a graphics card that uses High Bandwidth Memory. See through-silicon vias (TSV).

The HBM DRAM is tightly coupled to the host compute die with a distributed interface. The interface is divided into independent channels. The channels are completely independent of one another and are not necessarily synchronous to each other. The HBM DRAM uses a wide-interface architecture to achieve high-speed, low-power operation. The HBM DRAM uses a 500 MHz differential clock CK_t / CK_c (where the suffix "_t" denotes the "true", or "positive", component of the differential pair, and "_c" stands for the "complementary" one). Commands are registered at the rising edge of CK_t, CK_c. Each channel interface maintains a 128‑bit data bus operating at double data rate (DDR). HBM supports transfer rates of 1 GT/s per pin (transferring 1 bit), yielding an overall package bandwidth of 128 GB/s.[18]

HBM2

[edit]

The second generation of High Bandwidth Memory, HBM2, also specifies up to eight dies per stack and doubles pin transfer rates up to 2 GT/s. Retaining 1024‑bit wide access, HBM2 is able to reach 256 GB/s memory bandwidth per package. The HBM2 spec allows up to 8 GB per package. HBM2 is predicted to be especially useful for performance-sensitive consumer applications such as virtual reality.[19]

On January 19, 2016, Samsung announced early mass production of HBM2, at up to 8 GB per stack.[20][21] SK Hynix also announced availability of 4 GB stacks in August 2016.[22]

  • HBM2 DRAM die
    HBM2 DRAM die
  • HBM2 controller die
    HBM2 controller die
  • The HBM2 interposer of a Radeon RX Vega 64 GPU, with removed HBM dies; the GPU is still in place
    The HBM2 interposer of a Radeon RX Vega 64 GPU, with removed HBM dies; the GPU is still in place

HBM2E

[edit]

In late 2018, JEDEC announced an update to the HBM2 specification, providing for increased bandwidth and capacities.[23] Up to 307 GB/s per stack (2.5 Tbit/s effective data rate) is now supported in the official specification, though products operating at this speed had already been available. Additionally, the update added support for 12‑Hi stacks (12 dies) making capacities of up to 24 GB per stack possible.

On March 20, 2019, Samsung announced their Flashbolt HBM2E, featuring eight dies per stack, a transfer rate of 3.2 GT/s, providing a total of 16 GB and 410 GB/s per stack.[24]

August 12, 2019, SK Hynix announced their HBM2E, featuring eight dies per stack, a transfer rate of 3.6 GT/s, providing a total of 16 GB and 460 GB/s per stack.[25][26] On July 2, 2020, SK Hynix announced that mass production has begun.[27]

In October 2019, Samsung announced their 12-layered HBM2E.[28]

HBM3

[edit]

In late 2020, Micron unveiled that the HBM2E standard would be updated and alongside that they unveiled the next standard known as HBMnext (later renamed to HBM3). This was to be a big generational leap from HBM2 and the replacement to HBM2E. This new VRAM would have come to the market in the Q4 of 2022. This would likely introduce a new architecture as the naming suggests.

While the architecture might be overhauled, leaks pointed to performance similar to the updated HBM2E standard. This RAM was likely to be used mostly in data center GPUs.[29][30][31][32]

In mid 2021, SK Hynix unveiled some specifications of the HBM3 standard, with 5.2 Gbit/s I/O speeds and bandwidth of 665 GB/s per package, as well as up to 16-high 2.5D and 3D solutions.[33][34]

On 20 October 2021, before the JEDEC standard for HBM3 was finalised, SK Hynix was the first memory vendor to announce that it has finished development of HBM3 memory devices. According to SK Hynix, the memory would run as fast as 6.4 Gbit/s/pin, double the data rate of JEDEC-standard HBM2E, which formally tops out at 3.2 Gbit/s/pin, or 78% faster than SK Hynix's own 3.6 Gbit/s/pin HBM2E. The devices support a data transfer rate of 6.4 Gbit/s and therefore a single HBM3 stack may provide a bandwidth of up to 819 GB/s. The basic bus widths for HBM3 remain unchanged, with a single stack of memory being 1024-bits wide. SK Hynix would offer their memory in two capacities: 16 GB and 24 GB, aligning with 8-Hi and 12-Hi stacks respectively. The stacks consist of 8 or 12 16 Gb DRAMs that are each 30 μm thick and interconnected using Through Silicon Vias (TSVs).[35][36][37]

According to Ryan Smith of AnandTech, the SK Hynix first generation HBM3 memory has the same density as their latest-generation HBM2E memory, meaning that device vendors looking to increase their total memory capacities for their next-generation parts would need to use memory with 12 dies/layers, up from the 8 layer stacks they typically used until then.[35] According to Anton Shilov of Tom's Hardware, high-performance compute GPUs or FPGAs typically use four or six HBM stacks, so with SK Hynix's HBM3 24 GB stacks they would accordingly get 3.2 TB/s or 4.9 TB/s of memory bandwidth. He also noted that SK Hynix's HBM3 chips are square, not rectangular like HBM2 and HBM2E chips.[36] According to Chris Mellor of The Register, with JEDEC not yet having developed its HBM3 standard, might mean that SK Hynix would need to retrofit its design to a future and faster one.[37]

JEDEC officially announced the HBM3 standard on January 27, 2022.[8] The number of memory channels was doubled from 8 channels of 128 bits with HBM2e to 16 channels of 64 bits with HBM3. Therefore, the total number of data pins of the interface is still 1024.[38]

In June 2022, SK Hynix announced they started mass production of industry's first HBM3 memory to be used with Nvidia's H100 GPU expected to ship in Q3 2022. The memory will provide H100 with "up to 819 GB/s" of memory bandwidth.[39]

In August 2022, Nvidia announced that its "Hopper" H100 GPU will ship with five active HBM3 sites (out of six on board) offering 80 GB of RAM and 3 TB/s of memory bandwidth (16 GB and 600 GB/s per site).[40]

HBM3E

[edit]

On 30 May 2023, SK Hynix unveiled its HBM3E memory with 8 Gbit/s/pin data processing speed (25% faster than HBM3), which is to enter production in the first half of 2024.[41] At 8 GT/s with 1024-bit bus, its bandwidth per stack is increased from 819.2 GB/s as in HBM3 to 1 TB/s.

On 26 July 2023, Micron announced its HBM3E memory with 9.6 Gbit/s/pin data processing speed (50% faster than HBM3).[42] Micron HBM3E memory is a high-performance HBM that uses 1β DRAM process technology and advanced packaging to achieve the highest performance, capacity and power efficiency in the industry. It can store 24 GB per 8-high cube and allows data transfer at 1.2 TB/s. There will be a 12-high cube with 36 GB capacity in 2024.

In August 2023, Nvidia announced a new version of their GH200 Grace Hopper superchip that utilizes 141 GB (144 GiB physical) of HBM3e over a 6144-bit bus providing 50% higher memory bandwidth and 75% higher memory capacity over the HBM3 version.[43]

In May 2023, Samsung announced HBM3P with up to 7.2 Gbit/s which will be in production in 2024.[44]

On October 20, 2023, Samsung announced their HBM3E "Shinebolt" with up to 9.8 Gbit/s memory.[45]

On February 26, 2024, Micron announced the mass production of Micron's HBM3E memory.[46]

On March 18, 2024, Nvidia announced the Blackwell series of GPUs using HBM3E memory[47]

On March 19, 2024, SK Hynix announced the mass production of SK Hynix's HBM3E memory.[48]

In September 2024, SK Hynix announced the mass production of its 12-layered HBM3E memory[49] and in November the 16-layered version.[50]

HBM-PIM

[edit]

In February 2021, Samsung announced the development of HBM with processing-in-memory (PIM). This new memory brings AI computing capabilities inside the memory, to increase the large-scale processing of data. A DRAM-optimised AI engine is placed inside each memory bank to enable parallel processing and minimise data movement. Samsung claims this will deliver twice the system performance and reduce energy consumption by more than 70%, while not requiring any hardware or software changes to the rest of the system.[51]

HBM4

[edit]

In July 2024, JEDEC announced its preliminary specifications for future HBM4.[52] It lowered data rate per pin back to 6.4 Gbit/s/pin (the level of HBM3) but since it now employs a 2048-bit interface per stack (doubling that of the previous generations), it still achieves greater (1.6TB/s)[53] data rate per stack than that of HBM3E. Additionally, it will allow 4GB layers (yielding 64GB in 16-layer configurations).

In April 2025, JEDEC released the official HBM4 specification.[9] According to Kunal Khullar of Tom's Hardware, it supports transfer speeds of up to 8 Gb/s across a 2048-bit interface, with total bandwidth of up to 2 TB/s, and stack height of 4 to 16, with DRAM die densities of 24Gb or 32Gb, allowing for capacities up to 64GB. HBM4 is backwards compatible to HBM3 controllers. Samsung, Micron, and SK hynix contributed to the standard's formulation.[10]

History

[edit]

Background

[edit]

Die-stacked memory was initially commercialized in the flash memory industry. Toshiba introduced a NAND flash memory chip with eight stacked dies in April 2007,[54] followed by Hynix Semiconductor introducing a NAND flash chip with 24 stacked dies in September 2007.[55]

3D-stacked random-access memory (RAM) using through-silicon via (TSV) technology was commercialized by Elpida Memory, which developed the first 8 GB DRAM chip (stacked with four DDR3 SDRAM dies) in September 2009, and released it in June 2011. In 2011, SK Hynix introduced 16 GB DDR3 memory (40 nm class) using TSV technology,[3] Samsung Electronics introduced 3D-stacked 32 GB DDR3 (30 nm class) based on TSV in September, and then Samsung and Micron Technology announced TSV-based Hybrid Memory Cube (HMC) technology in October.[56]

JEDEC first released the JESD229 standard for Wide IO memory,[57] the predecessor of HBM featuring four 128 bit channels with single data rate clocking, in December 2011 after several years of work. The first HBM standard JESD235 followed in October 2013.

Development

[edit]
AMD Fiji, the first GPU to use HBM

The development of High Bandwidth Memory began at AMD in 2008 to solve the problem of ever-increasing power usage and form factor of computer memory. Over the next several years, AMD developed procedures to solve die-stacking problems with a team led by Senior AMD Fellow Bryan Black.[58] To help AMD realize their vision of HBM, they enlisted partners from the memory industry, particularly Korean company SK Hynix,[58] which had prior experience with 3D-stacked memory,[3][55] as well as partners from the interposer industry (Taiwanese company UMC) and packaging industry (Amkor Technology and ASE).[58]

The development of HBM was completed in 2013, when SK Hynix built the first HBM memory chip.[3] HBM was adopted as industry standard JESD235 by JEDEC in October 2013, following a proposal by AMD and SK Hynix in 2010.[6] High volume manufacturing began at a Hynix facility in Icheon, South Korea, in 2015.

The first GPU utilizing HBM was the AMD Fiji which was released in June 2015 powering the AMD Radeon R9 Fury X.[4][59][60]

In January 2016, Samsung Electronics began early mass production of HBM2.[20][21] The same month, HBM2 was accepted by JEDEC as standard JESD235a.[7] The first GPU chip utilizing HBM2 is the Nvidia Tesla P100 which was officially announced in April 2016.[61][62]

In June 2016, Intel released a family of Xeon Phi processors with 8 stacks of HCDRAM, Micron's version of HBM. At Hot Chips in August 2016, both Samsung and Hynix announced a new generation HBM memory technologies.[63][64] Both companies announced high performance products expected to have increased density, increased bandwidth, and lower power consumption. Samsung also announced a lower-cost version of HBM under development targeting mass markets. Removing the buffer die and decreasing the number of TSVs lowers cost, though at the expense of a decreased overall bandwidth (200 GB/s).

Nvidia announced Nvidia Hopper H100 GPU, the world's first GPU utilizing HBM3 on March 22, 2022.[65]

See also

[edit]

References

[edit]
[edit]
Revisions and contributorsEdit on WikipediaRead on Wikipedia

High Bandwidth Memory

View on Grokipedia
from Grokipedia
High Bandwidth Memory (HBM) is a high-performance (DRAM) technology that employs a 3D-stacked with through-silicon vias (TSVs) to deliver exceptionally high bandwidth and low power consumption compared to traditional DRAM interfaces like DDR or GDDR. Standardized by under specifications such as JESD235 for HBM and JESD235A for HBM2, it features a wide-interface design with multiple independent channels—typically eight channels of 128 bits each for a total 1024-bit bus—operating at (DDR) speeds to achieve bandwidths up to several terabytes per second per stack. Originating from a collaboration between and , the first HBM prototypes were developed in 2013 to address memory bandwidth bottlenecks in graphics processing units (GPUs), with producing the initial chips that year. JEDEC formally adopted the HBM standard in October 2013, and the technology debuted commercially in AMD's Fiji-series GPUs in 2015, marking the first widespread use of 3D-stacked memory in consumer hardware. Evolution continued with HBM2 in 2016, enhancing capacity and efficiency; HBM2E in 2020, offering up to 3.6 Gbps per pin and 460 GB/s bandwidth; HBM3 in 2022, with 6.4 Gbps speeds and on-die error correction for AI workloads; HBM3E in 2023, extending speeds to 9.6 Gbps for over 1.2 TB/s bandwidth in AI systems; and HBM4 finalized by JEDEC in April 2025, introducing architectural improvements for even higher bandwidth and power efficiency in next-generation systems; HBM4E, an enhanced variant announced in late 2025 by manufacturers such as Samsung, SK Hynix, and Micron, offering speeds up to 13 Gbps per pin and bandwidths of up to 3.25 TB/s per stack for advanced AI applications. HBM's defining advantages stem from its tightly coupled integration with host processors via interposers or advanced , enabling low-latency transfer ideal for bandwidth-intensive applications. It excels in GPUs for graphics rendering, (HPC) simulations, and (AI) training/inference, where parallel processing demands massive throughput—such as in NVIDIA's AI accelerators and supercomputers—while consuming less power per bit than alternatives like GDDR6. The expansion of AI data centers has caused explosive demand for HBM and other high-value memory products, with AI applications projected to consume approximately 20% of global DRAM wafer capacity by 2026 and HBM3E prioritized for hyperscalers, leading to production shortages and significant price surges in early 2026. Memory manufacturers prioritize HBM production due to its higher profitability in AI and GPU applications, leading companies like Samsung, SK Hynix, and Micron to shift production capacities from standard DRAM to HBM, which limits output for consumer uses and contributes to price increases in conventional memory. As AI and HPC demands surge, HBM's market is projected to expand significantly, driven by its role in enabling efficient handling of large datasets in multi-core environments. The rising demand for HBM, driven by AI applications, benefits semiconductor packaging companies through surging orders for HBM packaging, capacity utilization exceeding 90%, full production lines, and growth in advanced packaging technologies like XDFOI for high-density integration, enabling high demand elasticity, strong pricing power, and improved gross margins as a key midstream link in the supply chain.

Overview

Definition and Purpose

High Bandwidth Memory (HBM) is a high-speed interface standard for 3D-stacked (SDRAM), designed to deliver exceptional data throughput in performance-critical systems. Developed as a collaborative effort among industry leaders, HBM integrates multiple DRAM dies vertically using through-silicon vias (TSVs) to form compact stacks, enabling a wide interface that connects directly to processors via interposers. This architecture was formalized by the Solid State Technology Association in October 2013 through the JESD235 standard, aiming to overcome the bandwidth constraints of conventional technologies amid escalating demands from compute-intensive applications. The primary purpose of HBM is to alleviate the bottleneck in traditional DRAM configurations, where narrow buses and longer signal paths limit data transfer rates for parallel processing tasks. By providing ultra-high data rates—reaching up to terabytes per second—HBM supports workloads such as rendering, inference, and scientific simulations that require massive parallel data access. It is particularly suited for graphics processing units (GPUs) and specialized accelerators, where rapid data movement between memory and compute cores is essential for maintaining efficiency in environments. At its core, the 3D stacking approach in HBM minimizes latency by shortening interconnect distances between memory layers and the host die, while simultaneously boosting to pack more capacity into a smaller footprint without increasing the overall system size. This vertical integration contrasts with planar layouts, allowing for wider channels that enhance throughput without relying solely on transistor scaling. The 2013 JEDEC standardization was motivated by the need to extend bandwidth growth beyond the limitations of in traditional semiconductor scaling, fostering innovations in die-stacking to meet the evolving requirements of GPUs and accelerators in data-parallel applications.

Key Features and Benefits

High Bandwidth Memory (HBM) employs a wide bus interface, typically featuring 1024-bit channels in earlier generations and up to 2048-bit channels in advanced variants, enabling significantly higher data throughput compared to narrower bus architectures like those in traditional DRAM. This design is facilitated by through-silicon vias (TSVs), which provide high-density vertical interconnects between stacked DRAM dies, minimizing signal path lengths and supporting efficient 3D integration. Additionally, HBM incorporates a base logic die that handles functions such as test logic and can integrate error correction mechanisms, enhancing reliability in high-performance environments. The primary benefits of HBM stem from its , delivering up to 1-2 TB/s of bandwidth per stack, which represents 2-5 times the performance of GDDR6 in comparable GPU configurations. This elevated bandwidth supports demanding applications like AI training and by reducing memory bottlenecks. Power efficiency is another key advantage, with around 4-5 pJ/bit for transfers, lower than conventional memories due to reduced and optimized signaling. HBM's allows for multi-stack configurations, enabling systems to aggregate bandwidth across up to eight stacks for total throughputs exceeding 10 TB/s while maintaining a compact footprint. Packaging efficiency in HBM is achieved through the use of silicon interposers in assemblies, which facilitate direct, high-speed connections between the memory stack and logic dies, and emerging hybrid bonding techniques that enable bumpless, fine-pitch interconnections for improved density and thermal management. However, HBM incurs a significantly higher cost per bit than standard DDR DRAM due to its complex manufacturing, though this premium is justified for bandwidth-intensive, premium applications where space and power savings outweigh the expense.

Architecture

Stacked Design and Components

High Bandwidth Memory (HBM) employs a vertical stacking architecture to integrate multiple (DRAM) dies, ranging from 4 layers in early generations to up to 16 layers in HBM4, depending on the generation and capacity requirements, atop a base logic die within a compact 3D (IC) package. These DRAM dies are interconnected using through-silicon vias (TSVs), which provide high-density vertical electrical pathways, with approximately 5,000 TSVs per layer handling signals, power, and ground distribution. The base logic die, positioned at the bottom of the stack, serves as a buffer for data interfacing with the host processor and supports error-correcting code (ECC) functionality through dedicated parity bits, while optional integration of controller logic can be incorporated to manage memory operations. The stacking relies on micro-bump connections, featuring arrays of up to 6,303 bumps with a 55 μm pitch, to ensure reliable interlayer bonding and signal integrity between dies. For off-chip connectivity, the HBM stack mounts onto a silicon interposer in a 2.5D/3D IC packaging configuration, which routes high-speed signals to the processor while minimizing latency and enabling dense integration. This design achieves high memory density, with capacities scaling up to 64 GB per stack in HBM4 (as of 2025) through increased die layers and larger per-die capacities. The approximate density scaling follows the relation D≈Ndies×CdieD \approx N_{\text{dies}} \times C_{\text{die}}, where DD is total stack density, NdiesN_{\text{dies}} is the number of DRAM dies, and CdieC_{\text{die}} is the capacity per die; however, thermal dissipation constraints limit NdiesN_{\text{dies}} to 12–16 to prevent overheating within the fixed stack height of around 720–775 μm. In TSV fabrication, liners isolate the copper-filled vias, with advanced processes incorporating high-k materials to reduce and improve electrical performance across the stack. management is addressed through integrated spreaders and thermal vias or dummy bumps, which distribute evenly from the densely packed dies to the package lid, mitigating hotspots that could degrade reliability. Yield challenges in stacking arise from defect propagation across layers, necessitating known good die (KGD) testing at interim stages to verify functionality before assembly, achieving yields above 98% in mature processes. In HBM4, the base die can be customized for advanced features like integrated and interfaces, while hybrid bonding may replace micro-bumps for pitches below 10 μm in future implementations.

Interface and Data Transfer

High Bandwidth Memory (HBM) employs a wide interface architecture standardized by , featuring a data bus of 1024 bits in HBM1-HBM3 (divided into 8 channels of 128 bits or 16 channels of 64 bits) and 2048 bits in HBM4 (32 channels), with each channel supporting 128-bit or narrower sub-divisions depending on the generation. This design utilizes augmented by a reference voltage (VREF) for pseudo-differential operation, which enhances noise rejection while minimizing pin count and power. Receivers incorporate PVT-tolerant techniques, such as adaptive equalization and voltage referencing, to maintain across process variations, supply voltage fluctuations, and temperature extremes. The data transfer protocol in HBM separates the command and address buses, with dedicated row address () and column address (CA) lines that allow simultaneous issuance of row and column access commands for improved . Burst length is 2 clock cycles (BL2), transferring 256 bits per 128-bit channel (or 128 bits per 64-bit channel in HBM3) in a single burst to optimize throughput for high-demand workloads. Refresh operations are tailored for the stacked die structure, supporting per-bank or targeted refresh modes that reduce overhead compared to all-bank refreshes in traditional DRAM, thereby preserving availability in multi-die configurations. Bandwidth in HBM is determined by the formula: Bandwidth (GB/s)=data rate per pin (Gbps)×total pins across channels8\text{Bandwidth (GB/s)} = \frac{\text{data rate per pin (Gbps)} \times \text{total pins across channels}}{8} This equation converts the aggregate bit-rate to bytes per second, where the division by 8 accounts for 8 bits per byte; for instance, a 2 Gbps per pin rate across 1024 pins (HBM1-HBM3) yields 256 GB/s, or across 2048 pins (HBM4) yields 512 GB/s. To ensure signal integrity over the short, high-density interconnects, HBM implements on-die termination (ODT) with dynamic calibration, applying resistive termination at the receiver to match driver impedance and suppress reflections. Timing benefits from direct die-to-die paths via through-silicon vias (TSVs), enabling low-latency intra-stack operations with typical access latencies around 100 ns, benefiting from short die-to-die paths. The stacked design's proximity enables these low-latency paths.

Generations

HBM1

High Bandwidth Memory 1 (HBM1) represents the first generation of the HBM standard, formalized by the under JESD235 in October 2013. This specification introduced a high-performance DRAM architecture designed for applications requiring substantial data throughput, such as graphics processing units (GPUs). HBM1 stacks utilized through-silicon vias (TSVs) to interconnect multiple DRAM dies vertically, enabling a compact form factor with enhanced bandwidth compared to traditional planar DRAM configurations. The initial commercial production of HBM1 was achieved by in 2013, marking the debut of TSV-based stacking in mass-produced DRAM devices. The core specifications of HBM1 include a maximum stack capacity of 1 GB, achieved through a 4-high configuration of 2 Gbit dies (each contributing 256 MB). Each stack features eight independent 128-bit channels, supporting data transfer rates of up to 1 Gbps per pin. This results in a total bandwidth of approximately 128 GB/s per stack, calculated as 16 GB/s per channel across the eight channels (128 bits × 1 GT/s × 8 channels). The interface employs a wide I/O with differential clocking to facilitate low-power, high-speed operation, while the 2-channel per die layout optimizes inter-die communication via TSVs. HBM1's integration was first demonstrated in AMD's Fiji GPU architecture, released in , where four 1 GB stacks provided 512 GB/s aggregate bandwidth for high-end graphics workloads. At the channel level, HBM1 employs eight pseudo-channels per stack to manage access and interleaving, allowing independent addressing within each 128-bit sub-channel for improved parallelism. Error handling is limited to basic on-die detection mechanisms for single-bit faults and post-package repair capabilities, without support for full error-correcting code (ECC) to maintain simplicity and cost efficiency in the initial design. This architecture prioritizes bandwidth density over extensive redundancy, relying on TSVs for that reduces signal latency but introduces challenges in thermal management and alignment precision. Despite its innovations, HBM1 faced limitations in , capping at 1 GB per stack, which constrained for emerging memory-intensive applications relative to subsequent generations. Bandwidth was also modest at 128 GB/s per stack, insufficient for the escalating demands of later scenarios. Manufacturing complexity arose from the novel TSV processes and 3D stacking, leading to initial yield issues due to defects in via alignment and die bonding, which elevated production costs and limited early .

HBM2 and HBM2E

High Bandwidth Memory 2 (HBM2) represents the second generation of the HBM standard, standardized by in January 2016 under JESD235A. It builds on HBM1 by doubling the per-pin data rate to 2 Gbps while maintaining a 1024-bit wide interface divided into up to 8 independent 128-bit channels per stack. This configuration supports stack heights of 2 to 8 DRAM dies, with die densities from 1 Gb to 8 Gb, enabling capacities up to 8 GB per stack in an 8-high configuration. The resulting peak bandwidth reaches 256 GB/s per stack, calculated as the product of the pin speed, interface width, and channel count divided by 8 to convert bits to bytes. In contrast to HBM1's 1 Gbps per pin and maximum 128 GB/s per stack, HBM2's formula for bandwidth scaling is: BWHBM2=pin_speed×1024×channels8\text{BW}_{\text{HBM2}} = \frac{\text{pin\_speed} \times 1024 \times \text{channels}}{8} where pin_speed is in Gbps and channels range from 2 to 8, yielding up to twice the throughput of its predecessor for equivalent configurations. HBM2 also introduces full error-correcting code (ECC) support per channel for improved data integrity in high-reliability applications. Key enhancements in HBM2 focus on increased pin speeds achieved through advanced signaling techniques, such as pseudo-open drain I/O to reduce power consumption and improve at higher rates. It supports flexible channel configurations from 2 to 8, allowing scalability for diverse system needs, and operates at a core voltage of 1.2 V with I/O signaling optimized for efficiency, contributing to overall power gains over HBM1 despite the speed increase. These improvements enable HBM2 to deliver higher in bandwidth-intensive workloads while maintaining low latency and energy efficiency. HBM2E emerged as an evolutionary extension of HBM2 in 2019, driven by industry demands for greater capacity and speed without a full generational shift. It boosts per-pin data rates to 3.6–6.4 Gbps through refined and signaling, supporting up to 12-high stacks with up to 16 Gb dies (2 GB each) for capacities reaching 24 GB per stack. Bandwidth scales accordingly to up to 460 GB/s per stack at 3.6 Gbps, with higher rates possible in optimized implementations. Notable deployments include the A100 GPU, which utilizes HBM2E for 40–80 GB total memory and over 2 TB/s aggregate bandwidth across multiple stacks, and the MI250 accelerator with 128 GB HBM2E delivering 3.2 TB/s. HBM2E retains HBM2's ECC capabilities and channel flexibility, prioritizing seamless integration into existing HBM2 ecosystems for accelerated computing and AI systems.

HBM3 and HBM3E

High Bandwidth Memory 3 (HBM3) represents the third generation of the HBM standard, finalized by in January 2022 to address escalating demands for bandwidth in and applications. This iteration doubles the channel count to 16 channels (each 64 bits wide) for a 1024-bit interface per stack while supporting densities up to 24 GB in a 12-high configuration using 16 Gb DRAM layers. The base data rate operates at 6.4 Gbps per pin, delivering a peak bandwidth of up to 819 GB/s per stack, which significantly enhances data throughput for memory-intensive workloads. HBM3E serves as an energy-efficient extension to the HBM3 specification, with initial rollouts occurring in 2023 and broader adoption in , pushing per-pin speeds to 9.2–9.6 Gbps for improved without proportionally increasing power consumption. This variant achieves up to 1.2 TB/s bandwidth per stack and supports capacities reaching 36 GB, leveraging higher-density DRAM dies in multi-layer stacks. It has been integrated into advanced accelerators, such as NVIDIA's H200 GPU with 141 GB of HBM3E and AMD's Instinct MI325X with 256 GB capacity and 6 TB/s aggregate bandwidth, reflecting 2025 updates in AI hardware ecosystems. Key enhancements in HBM3 and HBM3E include adaptive refresh mechanisms, which dynamically adjust refresh intervals to reduce power usage during low-activity periods, and on-die (ECC) for improved reliability by detecting and correcting single-bit errors directly within the DRAM layers. Additionally, support for multi-stack daisy-chaining allows seamless interconnection of multiple HBM stacks, facilitating scalable configurations in large-scale systems without excessive signaling overhead. In practical operation, the effective throughput of HBM3 and HBM3E accounts for protocol and timing overheads, typically expressed as: \text{Effective throughput} = \text{base_BW} \times \text{efficiency_factor ($0.9$–$0.95$)} where base_BW is the theoretical peak bandwidth and the efficiency factor reflects real-world utilization, often around 85–95% in optimized AI scenarios.

Advanced Variants

High Bandwidth Memory (HBM) has seen innovative extensions through processing-in-memory (PIM) architectures, which integrate compute units directly into the memory stack to minimize data movement between processors and memory. Samsung developed HBM-PIM prototypes in 2023, embedding AI-dedicated processors within the HBM DRAM to offload operations like matrix multiplications, achieving up to 2x speedup in AI inference tasks such as GPT-J models. SK Hynix has similarly advanced PIM technologies since 2022, focusing on domain-specific memory for AI clusters. These variants reduce energy consumption by performing computations locally in memory; conceptually, the energy savings can be modeled as EPIM=Estandard×(1−compute locality)E_{\text{PIM}} = E_{\text{standard}} \times (1 - \text{compute locality}), where compute locality represents the fraction of operations executed in-memory, leading to reported reductions of up to 85% in data movement energy for transformer-based AI workloads. The next major advancement, HBM4, was standardized by JEDEC in April 2025 under JESD270-4, with development completed by major vendors such as SK Hynix in September 2025 and samples supplied to customers like NVIDIA; mass production is anticipated in 2026. SK Hynix is the primary supplier of HBM for NVIDIA's high-end AI GPUs, expected to hold approximately 70% market share for HBM4 in 2026. Micron serves as a significant secondary supplier with around 11% overall HBM market share in Q3 2025, while Samsung, holding about 35% share in the same period, is competing for larger market shares in NVIDIA's HBM4 contracts. It supports stack configurations up to 16-high using 24 Gb or 32 Gb DRAM dies for capacities reaching 64 GB per stack. It delivers over 2 TB/s bandwidth per stack via a 2048-bit interface at 8 Gbps per pin, with vendors like SK Hynix targeting over 10 Gbps for enhanced AI and high-performance computing applications. HBM4 incorporates hybrid bonding for finer interconnect pitches, enabling tighter integration with compute dies and reduced latency compared to prior generations. HBM4E represents an enhanced variant of HBM4, developed by manufacturers including Samsung, Micron, and SK Hynix to meet the escalating demands of AI and high-performance computing systems. It achieves per-pin data rates up to 13 Gbps, delivering bandwidths of up to 3.25 TB/s per stack, which is approximately 2.5 times higher than HBM3E. These advancements leverage hybrid bonding and other refined interconnect technologies for improved efficiency and integration. Mass production of HBM4E is anticipated starting in 2027, with samples already being supplied to partners like NVIDIA for next-generation accelerators. Emerging variants extend HBM's utility in disaggregated systems through integration with (CXL), allowing pooled HBM resources across servers for flexible memory allocation in AI clusters, as demonstrated in Samsung's 2023 prototypes combining HBM-PIM with CXL for up to 1.1 TB/s bandwidth and 512 GB capacity. Additionally, evolutions in packaging, including advanced silicon interposers and hybrid bonding, support higher-density HBM stacks with improved thermal management and for next-generation AI accelerators.

Historical Development

Origins and Background

The development of High Bandwidth Memory (HBM) originated in the from on three-dimensional integrated circuits (3D ICs), spearheaded by initiatives from the and academic institutions, aimed at overcoming the "memory wall" in von Neumann architectures. This memory wall, first articulated by Wulf and McKee, describes the widening gap where processor computational speeds have outpaced memory access latencies and bandwidth improvements by factors of 50 to 100, creating a bottleneck in data-intensive applications. 3D IC focused on vertically stacking components to shorten interconnects, reduce latency, and enhance bandwidth density, with early explorations dating back to DARPA-funded programs on heterogeneous integration in the early . Key early concepts for HBM's stacked architecture emerged from academic and industry papers in the mid-2000s, including IEEE publications proposing vertical interconnections for chip stacks to enable wider data paths and higher throughput in memory systems. For instance, a 2004 IEEE paper detailed process integration techniques for 3D chip stacks using through-silicon vias (TSVs) to facilitate dense vertical signaling, laying foundational ideas for memory-logic integration. Initial prototypes of stacked DRAM with wide interfaces, such as Samsung's Wide-I/O mobile DRAM, were demonstrated around 2011, building on these concepts to achieve preliminary high-bandwidth in lab settings. Driving this evolution were the escalating memory demands of GPU advancements post-2010, as and pushed architectures like Fermi and subsequent generations that amplified parallel compute but strained traditional GDDR memory's bandwidth limits in high-end graphics and emerging compute workloads. Power efficiency constraints in data centers further necessitated innovations like 3D stacking, as conventional memory interfaces consumed excessive energy for scaling bandwidth beyond 10 GB/s per channel. Precursor standards, such as the Wide I/O interface developed under with input from the , provided early frameworks for low-power, wide-channel 3D memory suitable for mobile and high-performance applications. In response to GDDR's limitations in power and scalability for ultra-high-end graphics, collaborated closely with starting in 2013 to pioneer HBM as a next-generation solution, emphasizing 3D stacking to deliver terabit-per-second bandwidth while maintaining compact form factors. SK Hynix has strategically focused on leadership in AI memory technology through its development of advanced HBM variants, essential for AI GPU accelerators. This industry partnership addressed the need for memory that could keep pace with GPU compute scaling without exacerbating energy demands. later contributed to HBM evolution through standardization and HBM2 production.

Standardization and Milestones

The standardization of High Bandwidth Memory (HBM) was spearheaded by the Joint Electron Device Engineering Council (JEDEC), which published the initial JESD235 specification in October 2013 to define the architecture and interface for HBM1. Key semiconductor manufacturers, including Samsung, SK Hynix, and Micron, contributed significantly to the development of this standard through their participation in JEDEC committees, ensuring compatibility across industry ecosystems. In January 2016, JEDEC released the updated JESD235A specification for HBM2, which enhanced data rates and capacity while maintaining backward compatibility with the original framework. The JESD238 standard for HBM3 followed in January 2022, introducing higher pin speeds up to 6.4 Gbps and support for up to 16 channels to meet escalating bandwidth demands in high-performance computing. A major milestone in HBM's adoption occurred in June 2015 with the launch of the R9 Fury X graphics card, the first commercial product to integrate HBM1, delivering 512 GB/s of bandwidth in a 4 GB stack. NVIDIA advanced this trajectory in 2017 by incorporating HBM2 into its Tesla V100 accelerator based on the Volta architecture, enabling 900 GB/s bandwidth for applications. In 2019, vendors like and introduced HBM2E as a non-JEDEC extension, boosting per-pin speeds to 3.6 Gbps and capacities up to 24 GB per stack to bridge gaps until full HBM3 ratification. HBM3E sampling began in 2023, with unveiling 8 Gbps/pin modules in May and Micron following with 24 GB 8-high stacks for NVIDIA's H200 GPUs. The from 2023 to 2025 propelled HBM's market growth, with the expanding from approximately $4 billion in 2023 to an estimated $35 billion in 2025, according to Micron's forecasts. This surge led to supply shortages in 2024 and 2025, as demand outpaced production; for instance, reported its HBM supply nearly sold out for 2025 due to NVIDIA's procurement needs. By 2025, HBM integration reached over 70% of top AI GPUs, driven by partnerships such as TSMC's CoWoS advanced , which facilitates efficient stacking of HBM with GPUs from and . In September 2025, completed development of the world's first HBM4, preparing for to support next-generation AI systems.

Applications

Graphics and Gaming

High Bandwidth Memory (HBM) has seen early adoption in graphics processing units (GPUs) primarily for high-end gaming and professional visualization applications, where its stacked architecture provides superior bandwidth compared to traditional GDDR memory. AMD integrated HBM2 with its in 2017 to deliver up to 483 GB/s of , which supported enhanced performance in demanding rendering tasks. This was followed by the VII in 2019, featuring 16 GB of HBM2 across a 4096-bit interface for 1 TB/s bandwidth, enabling smooth 4K and 8K video playback and gaming at high frame rates in titles requiring intensive graphical computations. In gaming scenarios, HBM's sustained high bandwidth excels at rapid texture loading and processing complex shaders, minimizing latency in real-time rendering pipelines. This is particularly beneficial for ray tracing workloads, where HBM facilitates quicker access to large datasets for light simulation and reflection calculations, resulting in more realistic visuals without frame drops. For virtual reality (VR) and augmented reality (AR) applications, HBM reduces memory bottlenecks during high-fidelity environment rendering, supporting immersive experiences with minimal stuttering in dynamic scenes. NVIDIA has also leveraged HBM in professional graphics cards, such as the GP100 released in 2017, which utilized 16 GB of HBM2 for bandwidth-intensive tasks like and in gaming development workflows. Although gaming GPUs have largely stuck to GDDR variants due to , HBM's power efficiency—achieving high throughput at lower voltages—has influenced designs akin to gaming consoles. Despite these advantages, HBM's higher costs restrict its use to premium GPUs, primarily in flagship models for enthusiasts and professionals. This premium positioning ensures HBM targets scenarios where bandwidth demands outweigh affordability concerns, such as ultra-high-resolution gaming and .

AI and High-Performance Computing

High Bandwidth Memory (HBM) provides high bandwidth data transmission for AI accelerators such as GPUs and TPUs, playing a pivotal role in (AI) accelerators, where its high bandwidth and capacity enable efficient handling of large-scale for and workloads. SK Hynix has established leadership in HBM technology for AI applications, dominating production of advanced versions such as HBM3E and contributing to the development of HBM4. For Nvidia's AI accelerators, major HBM suppliers include SK Hynix as the primary provider, Micron as a significant secondary supplier, and Samsung, which is in close discussions to supply larger volumes for next-generation HBM4. In NVIDIA's Hopper architecture GPUs, such as the H100 introduced in 2023 and the H200 in 2024, HBM3 and HBM3e provide up to 141 GB of memory per GPU, supporting the processing of massive large language models (LLMs) like those exceeding 100 billion parameters without extensive model sharding. This configuration delivers up to 4.8 TB/s of bandwidth, facilitating faster matrix multiplications critical for transformer-based architectures in LLM . Compared to prior generations using HBM2e, such as the A100, the H100 and H200 achieve 3x to 4x improvements in throughput for LLMs due to enhanced memory access speeds and tensor core optimizations. In high-performance computing (HPC), HBM integration in GPU-accelerated nodes supports exascale simulations requiring rapid data throughput for complex scientific computations. The Frontier supercomputer, deployed in 2022 at Oak Ridge National Laboratory, leverages AMD EPYC processors paired with Instinct MI250X GPUs equipped with 128 GB of HBM2e per accelerator, enabling peak performance of over 1.1 exaFLOPS for double-precision workloads. This setup has powered advanced climate modeling, including the SCREAM (Spectrally coupled Community Atmosphere Model with Emphasized Array Methods) simulation, which resolved global cloud processes at kilometer-scale resolution in under a day—advancing predictions of extreme weather patterns and their U.S. impacts. By 2025, HBM adoption extends to tensor processing units (TPUs) and custom application-specific integrated circuits (), addressing the demands of distributed AI paradigms like . Google's (TPU v6e), previewed in 2024 and scaling into production, doubles HBM capacity to 32 GB per chip with 1.64 TB/s bandwidth, enhancing efficiency for privacy-preserving federated training across edge devices and data centers. Custom ASICs from vendors like , integrated with HBM3e stacks, enable multi-terabyte memory pools in hyperscale clusters, reducing latency in collaborative model updates for federated scenarios. HBM's proximity to compute logic minimizes data movement overhead in AI pipelines, lowering energy costs for memory-bound operations and enabling sustainable scaling to exaFLOPS-level performance (10^15 FLOPS). The expansion of AI data centers has caused explosive demand for HBM and other high-value memory products, driven by AI advancements, straining production resources as manufacturing 1 GB of HBM requires approximately three times the silicon wafer capacity compared to standard DRAM, contributing to shortages and price increases in conventional RAM supplies. In HPC and AI systems, this architecture supports the bandwidth needs of trillion-parameter models, ensuring efficient resource utilization as compute clusters expand toward zettascale ambitions.

Comparisons and Future Outlook

Versus Other Memory Technologies

High Bandwidth Memory (HBM) offers substantial advantages in bandwidth over GDDR6 and GDDR6X, primarily due to its wide interface and , enabling a single HBM3E stack to achieve up to 1.2 TB/s, compared to approximately 1 TB/s total bandwidth in high-end GDDR6X implementations like NVIDIA's RTX 4090 GPU. This results in 3-5x higher effective bandwidth for bandwidth-intensive workloads, though GDDR6X remains preferable for cost-sensitive gaming applications where its lower price point—about 3-5x less per GB than HBM—offsets slightly reduced peak throughput. HBM also incurs 2-3x higher latency in low-load scenarios due to its lower per-pin clock speeds, but its proximity to the processor via integration mitigates this under sustained high utilization. In contrast to DDR5 and LPDDR5, HBM's vertical stacking yields roughly 10x greater bandwidth density, packing terabytes per second into a compact footprint that suits space-constrained high-performance systems, though it requires approximately three times more silicon wafer area per gigabyte than DDR5 due to stacking complexities and larger dies, while a typical DDR5 delivers only about 76.8 GB/s at 9.6 GT/s. DDR5 and LPDDR5, however, provide superior capacity scalability, with modules reaching up to 128 GB, and benefit from widespread adoption in and server platforms for their lower cost and simpler integration. HBM's , often 5x higher per GB, limits its use to specialized domains where bandwidth trumps volume. In the context of AI-driven demand, NAND flash focuses on backend storage for AI servers (e.g., high-capacity SSDs) and enterprise/consumer applications with moderate demand growth and mid-single-digit to low double-digit price rises, whereas HBM targets explosive AI training and inference needs with significantly stronger price surges and margin expansion.
MetricHBM3E (per stack)GDDR6X (high-end GPU total)DDR5 (per module)
Bandwidth1.2 TB/s1 TB/s76.8 GB/s
Power Consumption~30 W~35-50 W (total for 24 chips)~10 W
Cost ($/GB)$10-20$5-15$5-10
Modern GPU architectures frequently employ hybrid memory configurations, utilizing HBM as a high-speed L2 cache for compute-critical tasks while relying on GDDR as the primary main memory for larger, less bandwidth-demanding storage needs, balancing performance and economics in designs from and . In these hierarchies, on-chip SRAM serves as low-level caches (L1-L3) for ultra-low latency access, but its high cost—upwards of $5,000 per GB due to dense transistor requirements—makes it impractical for large capacities. HBM, costing 3x or more per GB than standard DDR5 but far less than SRAM, provides a cost-effective high-bandwidth solution for tens of GB in AI and GPU applications, enabling balanced speed, capacity, and economics. The High Bandwidth Memory (HBM) market is poised for substantial expansion, with projections estimating a value of tens to over $100 billion by 2030, fueled predominantly by workloads that are expected to drive over 55% of demand through high-bandwidth requirements exceeding 500 GB/s. AI data center expansion is causing explosive demand for high-bandwidth memory (HBM) and other high-value memory products, contributing to this growth and exacerbating supply chain constraints. This growth reflects a compound annual rate of approximately 30% for AI-focused HBM through the decade, as major hyperscalers and chipmakers prioritize memory solutions for training large language models and inference tasks. HBM4 advancements are central to this trajectory, enabling and 3D system-in-package integrations that support denser, more efficient multi-die architectures for next-generation accelerators. Key challenges in HBM development include constraints, where (TSV) yields for high-stack HBM4 prototypes have improved to nearly 80% as of late 2025 (from around 65% in mid-2025), though scalable production remains limited. throttling in dense stacks exacerbates these issues, as increases and heat dissipation demands, necessitating advanced cooling like liquid systems to maintain performance without speed reductions. Standardization efforts for HBM4, finalized by in April 2025, have seen vendor-specific delays due to yield and validation hurdles, pushing mass production timelines into 2026 for leading vendors, with some like Micron delayed to 2027. Future directions for HBM emphasize hybrid integrations to overcome bandwidth walls, including emerging optical interconnects that could enhance AI system scalability by reducing latency in memory access, with prototypes demonstrating feasibility for deployment in the late 2020s. Processing-in-memory (PIM) capabilities are gaining traction in HBM designs for AI chips, projected to grow at a 35% CAGR through 2033 by embedding compute logic directly in memory to mitigate von Neumann bottlenecks. Samsung Electronics is pursuing the development of glass substrate technology for next-generation HBM through in-house efforts led by Samsung Electro-Mechanics, combined with strategic partnerships and investments. This includes a memorandum of understanding signed on November 5, 2025, to establish a joint venture with Sumitomo Chemical for the production of glass core materials used in advanced package substrates, aimed at improving thermal and mechanical properties for high-performance computing applications. Samsung has also partnered with Jungwoo M-Tech (JWMT) for glass substrate processing equipment, acquiring a stake through its venture investment arm to support pilot line development at its Sejong plant, with mass production targeted for late 2026. Additionally, investments in Extol for metal plating technology and ongoing discussions with Chemtronics for glass substrate supply are part of this approach to enhance packaging efficiency for HBM4 and beyond. While (HMC) offers an alternative for niche , HBM's broader ecosystem adoption positions it as the dominant technology, with HMC maintaining only a supplementary role in specialized networking applications. The rising demand for HBM, driven by AI applications, has significantly benefited semiconductor packaging companies as a key midstream link in the supply chain. This demand has led to surging orders for packaging services related to DRAM, NAND Flash, and HBM, with capacity utilization rates exceeding 90% in 2025 and continuing into 2026, resulting in full production lines and rapid expansion efforts. AI-driven needs have spurred growth in advanced packaging technologies, such as XDFOI (X Dimension Fan-Out Integration), which enables high-density integration for AI and high-performance computing applications including GPUs. As midstream providers, packaging companies enjoy high demand elasticity and strong pricing power, allowing them to transmit upstream price increases—such as the 50-55% rise in DRAM and HBM prices in early 2026—downstream, thereby improving gross margins amid ongoing shortages. Economically, and Micron command over 80% of HBM supply in 2025, with at 59% and Micron at approximately 20%, creating a concentrated market vulnerable to disruptions. As of 2026, the leading companies in the HBM market are SK Hynix, which holds the market leadership position with over 60% share, followed by Micron and Samsung. AI demand surges in 2024 and 2025 have triggered severe pricing volatility and shortages, with HBM and DRAM prices rising over 100% year-over-year as of late 2025 amid sold-out allocations through 2026. Major DRAM vendors prioritize HBM production over standard DRAM because it yields higher profit margins (approximately 60% for HBM vs. 40% for standard DRAM) and meets surging demand from AI chipmakers like NVIDIA, consuming advanced fabrication capacity that could otherwise support standard DDR4/DDR5 modules. This prioritization is driven by the profitability of HBM for AI and GPU applications, leading companies like Samsung, SK Hynix, and Micron to shift production capacities from standard DRAM to HBM, often on a three-to-one basis where producing one bit of HBM forgoes three bits of conventional memory, thereby limiting output for consumer and standard uses and causing significant price increases in those markets. For instance, Micron has discontinued portions of its consumer PC memory business to redirect supply toward AI chips and servers, resulting in HBM production being sold out for all of 2026 and DRAM prices surging 50-55% in early 2026 compared to the previous quarter. This shift has boosted profitability, with Micron's net income nearly tripling in its most recent quarter and its stock rising 247% over the past year. However, despite strong bookings for HBM, as competition in the HBM market intensifies and DDR5 experiences extreme price surges driven by shortages inflating prices and margins, manufacturers may reallocate production capacity from HBM to DDR5 modules to capitalize on superior profitability per wafer or production line, even amid competitive pressures on HBM returns; for example, Samsung has shifted focus from HBM to DDR5 production. This situation is exacerbated by the resource-intensive nature of HBM production, where 1 GB of HBM requires up to four times the silicon wafer capacity compared to standard DRAM, thereby constraining overall DRAM production and contributing to shortages in conventional RAM for PCs, servers, and other applications.

References

  1. https://www.jedec.org/standards-documents/docs/jesd235a
  2. https://semiengineering.com/knowledge_centers/memory/volatile-memory/dynamic-random-access-memory/high-bandwidth-memory/
  3. https://www.intel.com/content/www/us/en/docs/programmable/683189/23-4-19-6-1/introduction-to-high-bandwidth-memory.html
  4. https://news.skhynix.com/the-story-of-sk-hynixs-hbm-development/
  5. https://www.jedec.org/standards-documents/docs/jesd238b01
  6. https://www.jedec.org/standards-documents/docs/jesd270-4
  7. https://wccftech.com/samsung-hbm4e-set-to-deliver-a-significant-bandwidth/
  8. https://www.tomshardware.com/pc-components/dram/hbm-undergoes-major-architectural-shakeup-as-tsmc-and-guc-detail-hbm4-hbm4e-and-c-hbm4e-3nm-base-dies-to-enable-2-5x-performance-boost-with-speeds-of-up-to-12-8gt-s-by-2027
  9. https://www.driehaus.com/perspectives/High-Bandwidth-Memory-Technology-for-AI-Applications
  10. https://www.trendforce.com/news/2025/12/26/news-ai-reportedly-to-consume-20-of-global-dram-wafer-capacity-in-2026-hbm-gddr7-lead-demand/
  11. https://www.cnbc.com/2026/01/10/micron-ai-memory-shortage-hbm-nvidia-samsung.html
  12. https://arstechnica.com/gadgets/2026/01/samsung-sk-hynix-and-micron-post-record-profits-as-ai-drives-up-ram-prices/
  13. https://www.eetimes.com/high-profit-margins-drive-memory-makers-to-prioritize-hbm-production/
  14. https://finance.yahoo.com/news/samsung-diverts-production-lucrative-ai-2026-01-07/
  15. https://www.idtechex.com/en/research-article/demand-for-hbm-to-grow-15-fold-by-2035-for-hpc-and-ai-says-idtechex/32837
  16. https://www.uncoveralpha.com/p/2026-ai-landscape-who-benefits-the
  17. https://www.pwc.com/gx/en/industries/technology/pwc-semiconductor-and-beyond-2026-full-report.pdf
  18. https://www.trendforce.com/news/2023/08/22/tsmcs-cowos-dominance-and-traditional-test-assembly-responses/
  19. https://www.synopsys.com/blogs/chip-design/designing-hbm3-high-bandwidth-memory-interfaces.html
  20. https://www.rambus.com/blogs/hbm3-everything-you-need-to-know/
  21. https://anysilicon.com/high-bandwidth-memory-white-paper/
  22. https://iedm23.mapyourshow.com/mys_shared/iedm23/handouts/15-6_Tue_14482.pdf
  23. https://newsletter.semianalysis.com/p/scaling-the-memory-wall-the-rise-and-roadmap-of-hbm
  24. https://assets.micron.com/adobe/assets/urn:aaid:aem:275edf31-79e3-4b6c-8bbd-a233babe9281/renditions/original/as/micron-hbm2e-memory-wp.pdf
  25. https://semiengineering.com/hbm-options-increase-as-ai-demand-soars/
  26. https://www.viksnewsletter.com/p/why-is-hbm-so-hard-to-manufacture
  27. https://eureka.patsnap.com/report-hbm4-tsv-engineering-resistance-capacitance-and-signal-quality
  28. https://eureka.patsnap.com/report-hbm4-thermal-path-design-and-hot-spot-mitigation-in-advanced-stacks
  29. https://www.linkedin.com/posts/kevin-cho-2845b1b9_hbm-thermalmanagement-skhynix-activity-7358064264344948737-Wsoa
  30. https://www.formfactor.com/blog/2024/high-bandwidth-memory-testing-a-key-component-of-advanced-packaging-new-video/
  31. https://www.jedec.org/news/pressreleases/jedec%C2%AE-and-industry-leaders-collaborate-release-jesd270-4-hbm4-standard-advancing
  32. https://www.jedec.org/sites/default/files/docs/JESD235D.pdf
  33. https://www.design-reuse.com/article/60926-design-considerations-for-high-bandwidth-memory-controller/
  34. https://www.researchgate.net/publication/383971948_High-Bandwidth_and_Energy-Efficient_Memory_Interfaces_for_the_Data-Centric_Era_Recent_Advances_Design_Challenges_and_Future_Prospects
  35. https://www.jedec.org/sites/default/files/docs/JESD235.pdf
  36. https://ieeexplore.ieee.org/document/6757501
  37. https://www.wevolver.com/article/what-is-hbm-high-bandwidth-memory-deep-dive-into-architecture-packaging-and-applications
  38. https://www.jedec.org/news/pressreleases/jedec-updates-groundbreaking-high-bandwidth-memory-hbm-standard
  39. http://monitorinsider.com/HBM.html
  40. https://semiengineering.com/the-challenges-of-designing-an-hbm2-phy/
  41. https://news.skhynix.com/sk-hynix-starts-mass-production-of-high-speed-dram-hbm2e/
  42. https://www.tomshardware.com/reviews/glossary-hbm-hbm2-high-bandwidth-memory-definition,5889.html
  43. https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/a100/pdf/nvidia-a100-datasheet-nvidia-us-2188504-web.pdf
  44. https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/instinct-mi200-datasheet.pdf
  45. https://semiengineering.com/hbm2e-the-e-stands-for-evolutionary/
  46. https://www.jedec.org/news/pressreleases/jedec-publishes-hbm3-update-high-bandwidth-memory-hbm-standard
  47. https://www.phoronix.com/news/JEDEC-HBM3
  48. https://www.micron.com/products/memory/hbm/hbm3e
  49. https://www.nvidia.com/en-us/data-center/h200/
  50. https://www.amd.com/en/products/accelerators/instinct/mi325x.html
  51. https://semiengineering.com/the-power-of-hbm3-memory-for-ai-training-hardware/
  52. https://www.synopsys.com/glossary/what-is-high-bandwitdth-memory-3.html
  53. https://www.wevolver.com/article/what-is-high-bandwidth-memory-3-hbm3-complete-engineering-guide-2025
  54. https://semiconductor.samsung.com/news-events/tech-blog/hbm-pim-cutting-edge-memory-technology-to-accelerate-next-generation-ai/
  55. https://www.hc2023.hotchips.org/assets/program/conference/day1/PIM/23_HC35_PIM_PNM_Samsung_final.pdf
  56. https://www.mk.co.kr/en/business/11338083
  57. https://www.jedec.org/news/pressreleases/jedec%25C2%25AE-and-industry-leaders-collaborate-release-jesd270-4-hbm4-standard-advancing
  58. https://finance.yahoo.com/news/sk-hynix-completes-worlds-first-233900809.html
  59. https://www.trendforce.com/news/2024/07/15/news-jedec-releases-new-hbm4-spec-as-memory-giants-gear-up-to-take-the-lead/
  60. https://news.skhynix.com/2026-market-outlook-focus-on-the-hbm-led-memory-supercycle/
  61. https://www.reuters.com/world/asia-pacific/samsung-electronics-highlights-progress-hbm4-chip-supply-2026-01-02/
  62. https://www.allaboutcircuits.com/news/jedec-officially-releases-hbm4-memory-standard/
  63. https://videocardz.com/newz/sk-hynix-preparing-hbm4-for-mass-production-10-gb-s-per-pin-above-jedecs-8-gb-s-specs
  64. https://ontoinnovation.com/resources/bridging-performance-and-yield-the-evolving-role-of-interconnect-technologies-in-hbm/
  65. https://www.trendforce.com/news/2025/09/24/news-micron-counters-hbm4-speed-doubts-with-11-gbps-custom-hbm4e-due-2027-with-higher-margins/
  66. https://www.tweaktown.com/news/109495/sk-hynix-samsung-and-micron-fighting-for-nvidia-supply-contracts-for-new-16-hi-hbm4-orders/index.html
  67. https://www.businesskorea.co.kr/news/articleView.html?idxno=254083
  68. https://www.igorslab.de/en/samsung-ends-afterburner-hbm4e-with-325-tb-s-bandwidth-to-exceed-nvidias-requirements/
  69. https://semiengineering.com/the-evolution-of-hbm/
  70. https://tspasemiconductor.substack.com/p/the-packaging-evolution-trilogy-hybrid
  71. https://dl.acm.org/doi/10.1145/216585.216588
  72. https://www.researchgate.net/publication/302431680_Research_and_Development_History_of_Three-Dimensional_Integration_Technology
  73. https://ieeexplore.ieee.org/document/1319400
  74. https://www.eeworldonline.com/samsung-wide-io-memory-for-mobile-products-a-deeper-look/
  75. https://www.nvidia.com/content/gtc-2010/pdfs/2275_gtc2010.pdf
  76. https://csl.stanford.edu/~christos/publications/2012.lpddr2.isca.pdf
  77. https://semiengineering.com/knowledge_centers/standards-laws/standards/wide-io-2/
  78. https://news.skhynix.com/exploring-the-ai-ecosystem-how-sk-hynixs-industry-leading-memory-fuels-ai-innovation/
  79. https://sst.semiconductor-digest.com/2013/12/amd-and-hynix-announce-joint-development-of-hbm-memory-stacks/
  80. https://www.koreajoongangdaily.joins.com/news/2025-09-02/business/tech/How-HBM-split-the-paths-of-Samsung-AMD-and-SK-hynix/2387442
  81. https://www.edn.com/a-sneak-peek-at-the-hbm-cold-war-between-samsung-and-sk-hynix/
  82. https://blocksandfiles.com/2025/06/26/four-more-hbm-generations/
  83. https://www.edn.com/jedec-finalizes-hbm4-standard/
  84. https://www.hpcwire.com/off-the-wire/jedec-publishes-hbm3-updated-to-high-bandwidth-memory-hbm-standard/
  85. https://ir.amd.com/news-events/press-releases/detail/619/amd-ushers-in-a-new-era-of-pc-gaming-with-radeontm-r9-and-r7-300-series-graphics-line-up-including-worlds-first-graphics-family-with-revolutionary-hbm-technology
  86. https://www.top500.org/news/an-early-debut-for-nvidias-volta-gpu/
  87. https://news.skhynix.com/corporate/fact-sheet/
  88. https://www.nextplatform.com/2025/06/30/skyrocketing-hbm-will-push-micron-through-10-billion-and-beyond/
  89. https://www.reuters.com/technology/nvidia-supplier-sk-hynix-says-hbm-chips-almost-sold-out-2025-2024-05-02/
  90. https://tspasemiconductor.substack.com/p/the-infinite-ai-compute-loop-hbm
  91. https://medium.com/%40Elongated_musk/the-next-five-years-of-memory-and-why-it-will-decide-ais-pace-27c4318fe963
  92. https://news.skhynix.com/sk-hynix-completes-worlds-first-hbm4-development-and-readies-mass-production/
  93. https://www.amd.com/en/newsroom/press-releases/2017-7-30-amd-redefines-the-enthusiast-gaming-experience-wit.html
  94. https://www.digitaltrends.com/computing/amd-vega-architecture-details-ces-2017/
  95. https://www.techpowerup.com/gpu-specs/radeon-vii.c3358
  96. https://www.supermicro.com/en/glossary/hbm4
  97. https://blog.imaginationtech.com/why-gamers-and-developers-should-care-about-ray-tracing/
  98. https://www.leadtek.com/eng/products/workstation_graphics%282%29/nvidia_quadro_gp100%2810776%29/detail
  99. https://news.skhynix.com/sk-hynix-showcases-hbm4-at-supercomputing-2025/
  100. https://www.businesskorea.co.kr/news/articleView.html?idxno=228000
  101. https://nvidianews.nvidia.com/news/nvidia-supercharges-hopper-the-worlds-leading-ai-computing-platform
  102. https://developer.nvidia.com/blog/nvidia-hopper-architecture-in-depth/
  103. https://acecloud.ai/blog/hbm3e-nvidia-h200-ai-training-inference/
  104. https://www.clarifai.com/blog/nvidia-h100
  105. https://www.ornl.gov/news/frontier-supercomputer-debuts-worlds-fastest-breaking-exascale-barrier
  106. https://www.hpcwire.com/2023/04/07/frontier-supercomputer-powers-scream-climate-model/
  107. https://cloud.google.com/blog/products/compute/trillium-sixth-generation-tpu-is-in-preview
  108. https://www.prnewswire.com/news-releases/tech-forum-2025-asics-packaging-and-hbm-reshape-the-ai-chip-race-302563856.html
  109. https://www.wevolver.com/article/tpu-vs-gpu-in-ai-a-comprehensive-guide-to-their-roles-and-impact-on-artificial-intelligence
  110. https://finance.yahoo.com/news/ai-data-centers-are-eating-the-memory-industry-and-it-could-hurt-your-wallet-130814928.html
  111. https://www.tomshardware.com/pc-components/ram/hbm-is-eating-your-ram
  112. https://www.tomshardware.com/tech-industry/a-deeper-look-at-the-tightened-chipmaking-supply-chain-and-where-it-may-be-headed-in-2026-nobodys-scaling-up-says-analyst-as-industry-remains-conservative-on-capacity
  113. https://globalsemiresearch.substack.com/p/the-ai-driven-memory-supercycle-surge
  114. https://nfina.com/sram-vs-dram/
  115. https://newsletter.semianalysis.com/p/the-memory-wall
  116. https://www.business-standard.com/technology/tech-news/ai-memory-shortage-hbm-ram-prices-surge-nvidia-micron-samsung-126011100109_1.html
  117. https://sourceability.com/post/the-memory-market-tightens-under-ai-demand
  118. https://www.reuters.com/world/asia-pacific/sk-hynix-expects-ai-memory-market-grow-30-year-2030-2025-08-11/
  119. https://www.digitimes.com/news/a20250724PD223/samsung-hbm4-production-2026-sk-hynix.html
  120. https://www.digitimes.com/news/a20251021PD210/samsung-dram-hbm4-production.html
  121. https://tecknexus.com/sk-hynix-hbm4-worlds-first-hbm4-chip/
  122. https://www.digitimes.com/news/a20251104PD230/micron-hbm4-samsung-sk-hynix-nvidia.html
  123. https://www.allpcb.com/allelectrohub/photonic-interconnects-aim-to-solve-ai-memory-bottlenecks
  124. https://www.archivemarketresearch.com/reports/processing-in-memory-ai-chips-838289
  125. https://www.samsungsem.com/global/newsroom/news/view.do?id=9850
  126. https://news.nate.com/view/20251210n01008
  127. https://www.youtube.com/watch?v=B4nrZBdligo
  128. https://x.com/Jukanlosreve/status/1972931344717271372
  129. https://www.openpr.com/news/4095280/hmc-hbm-market-share-driven-by-data-center-expansion
  130. https://www.axtekic.com/news/ase-2026-price-increase%3A-ai+driven-advanced-packaging.html
  131. https://www.statschippac.com/applications-computing
  132. https://www.techinsights.com/outlook-reports-2026/advanced-packaging-outlook-report
  133. https://www.trendforce.com/news/2025/09/29/news-breaking-the-memory-wall-hbm-basics-and-the-rise-of-hbm4-in-ai/
  134. https://finance.yahoo.com/news/3-semiconductor-stocks-to-play-the-ai-supercycle-according-to-analysts-203840729.html
  135. https://en.unibetter-ic.com/top-high-bandwidth-memory-companies-in-the-world/
  136. https://www.astutegroup.com/news/general/sk-hynix-holds-62-of-hbm-micron-overtakes-samsung-2026-battle-pivots-to-hbm4/
  137. https://www.reuters.com/world/china/chip-crunch-how-ai-boom-is-stoking-prices-less-trendy-memory-2025-10-20/
  138. https://www.digitimes.com/news/a20251208PD214/samsung-hbm-ddr5-dram-capacity.html
  139. https://www.trendforce.com/presscenter/news/20251218-12843.html
  140. https://www.tweaktown.com/news/109259/samsung-shifts-focus-from-hbm-to-ddr5-modules-ddr5-ram-results-in-far-more-profits-than-hbm/index.html
  141. https://www.tomshardware.com/pc-components/dram/dram-prices-surge-171-percent-year-over-year-ai-demand-drives-a-higher-yoy-price-increase-than-gold
  142. https://www.pcgamer.com/hardware/memory/were-facing-what-has-never-happened-before-hdd-dram-hbm-nand-all-in-severe-shortage-in-2026-says-silicon-motions-ceo-as-memory-prices-continue-to-soar/
Add your contribution
Related Hubs
Contribute something
User Avatar
No comments yet.