Samsung bets on PIM while SK hynix keeps eye on HBM

The race for next-generation AI memory is heating up.

Samsung Electronics is betting on processing-in-memory (PIM) chips to succeed high bandwidth memory (HBM), while SK hynix is working to keep HBM as the standard to cement its dominant position in the memory market.

Samsung Electronics unveiled detailed specifications for its Lpddr5x-PIM, a commercial PIM product based on low-power memory, for the first time at Hot Chips, an annual semiconductor conference, at Stanford University on Aug. 25. The inference-focused memory chip is designed for mobile devices with on-device AI, such as smartphones and tablet computers.

A conventional memory chip sends stored data to a processor, such as a CPU or GPU, for processing, then receives results back. But bottlenecks during the process can slow down data processing and increase power consumption.

PIM, however, can perform simple calculations directly through computing units built into the memory. By minimizing data transfers between the memory and processor, PIM can reduce heat and power consumption while allowing CPUs and GPUs to focus on more complex calculations, improving overall processing efficiency.

“HBM currently serves as the de facto standard for AI memory, but as AI shifts from training toward inference and expands into smaller devices such as smartphones, demand is growing for new memory that consumes less power,” Samsung Electronics said.

Using PIM in on-device AI products could double processing speeds compared to conventional low-power memory and triple throughput, which is measured in tokens processed per second, according to the company.

Meanwhile, its rival SK hynix is taking a different route by improving packaging technology to push HBM further.

SK hynix presented hybrid bonding and iHBM as next-generation technologies — designed to tackle HBM’s heat problem — at the conference.

Currently, HBM stacks up to 16 memory chips, and pushing that to 20 will require better heat control. One solution is hybrid bonding, which directly bonds the insulating materials and copper on semiconductor wafers. By eliminating the tiny gaps between chips, the technology allows denser stacking while reducing signal delays and heat.

SK hynix said that hybrid bonding can make each chip 24 percent thicker while cutting thermal resistance by 35 percent.

iHBM takes a more targeted approach by managing specific hot spots inside HBM. Dedicated pathways are created near areas where heat tends to build, such as around connections carrying signals between chips. A special material that conducts heat but not electricity allows heat to escape through those pathways.

The iHBM can reduce HBM’s thermal resistance by more than 30 percent, according to SK hynix.

But experts expect next-generation memory technologies to complement HBM rather than replace it.

HBM offers strong performance but is expensive and consumes large amounts of power, meaning that its use will likely remain focused on data centers. But other types of memory could specialize in AI inference.

Micron and fabless chipmaker Primemas earlier presented Compute Express Link (CXL) as an alternative to HBM at the memory chip exhibition FMS 2026. CXL acts as a type of external memory that allows CPUs to access dynamic random-access memory, or DRAM, located outside a server.

“Korean chipmakers took an early lead in the HBM market, but Chinese semiconductor companies, including [CXMT], are growing at a frightening pace,” Kim Yong-seok, a distinguished professor of semiconductor engineering at Gachon University, said. “To maintain Korea’s current share of the AI chip market, the industry also needs to accelerate the development of next-generation AI memory technologies such as PIM and CXL.”

BY KIM KYUNG-MI [[email protected]]