Micron unveils absurdly fast 256GB DDR5-8800 memory sticks

Micron unveiled its new 256 GB MCR DIMM memory modules at the GPU Technology Conference (GTC) hosted by Nvidia. We're going to give a nerdy breakdown on how this technology functions.

Targetted for next-generation servers, these modules are designed to provide a large capacity and performance boost performance boost making them ideal for AI and big data processing. The new 256 GB MCRDIMMs are being sampled with its customers, Micron confirmed earlier this week.

At the demonstration, Micron showed a tall version of the 256 GB DDR5-8800 memory stick while stating that standard height sticks will also be offered. Micron's 128GB DDR5-8000 RDIMM consumes around 10W at DDR5-4800, whereas the tall version consumes 20W.

The standard memory stick can be subject to overheating due to its 2Hi stacked packages, which means less space for thermal dissipation. The tall stick however has 80 DRAM chips on both sides of the module and won’t face such issues.

MCR) DIMMs have a specialized buffer that enables two physical ranks to work as two separate modules operating parallel to each other. This doubles the performance by simultaneously retrieving 128 bytes of data from both ranks per clock, doubling the performance of a single module. At the same time, the buffer works at speeds beyond the specified 8800 MT/s by working with its host memory controller using the DDR5 protocol.

Meanwhile, the buffer works with its host memory controller using the DDR5 protocol, albeit at speeds beyond those specified by the standard, at 8800 MT/s in this case.

Usually, a host memory controller (or CPU) is limited to fetching 64 bytes of data at a time. But unlike a typical module with two physical ranks the MCRDIMMS works by doubling that, greatly increasing the performance and capacity.

We understand that due to its demonstration at the GTC, the 256 GB MCRDIMMs are planned for AI servers based on Intel's Xeon Scalable 'Granite Rapid' processors. The 12-channel memory subsystem of Intel’s 'Granite Rapid' processors will be able to accommodate 3 TB of DDR5 memory by using 12 slots and a total of 6 TB of DRAM using 24 available slots.

PCs continue to get faster as consoles struggle to keep up frame rates, despite getting upgrades.