Gone In A Flash: Generative AI and other LLM-based tools are facing a significant data bandwidth challenge. Computational power continues to grow exponentially, but the massive amounts of data required for AI training and inference still has to pass through interfaces and storage technologies with limited bandwidth capabilities. Kioxia is proposing a novel solution by bringing NAND flash memory closer to the GPU.
Japanese memory manufacturer Kioxia is bringing a new class of super-fast SSDs to the enterprise market. The Kioxia GP1 Series is based on the PCIe 6.0 interface and can leverage the NVMe 2.2 protocol to serve as a middle ground between "pure" NVMe storage solutions and high-bandwidth memory VRAM soldered directly onto a GPU or AI accelerator's PCB.
The GP1 Super High IOPS SSDs can achieve 10 million input/output operations per second, Kioxia explains, thanks to the company's second-generation XL-FLASH memory. Unlike traditional TLC memory chips used in consumer products, XL-FLASH is designed to deliver higher IOPS performance with 512-byte block sizes. Furthermore, Kioxia is already working on future XL-FLASH generations that could increase data rates to 100 million IOPS.
The GP1 SSDs are optimized for direct GPU access through the PCIe interface. The drives should improve computational efficiency by allowing AI accelerators to quickly access larger datasets stored on flash memory, without requiring the massive expense of expanding the HBM memory pool attached to the GPU's PCB.
The new SSDs are available in E3.S and E1.S 9.5mm/15mm form factors, with support for either traditional air-cooling solutions or cold-plate liquid cooling (available only for E3.S and E1.S 9.5mm models). The drives offer storage endurance of up to 50 drive writes per day.
According to Neville Ichhaporia, senior vice president and general manager of the SSD unit at Kioxia America, XL-FLASH memory can deliver very high performance and low-latency operation, allowing it to serve as an effective memory extension tier for server GPUs. The cost per gigabyte is significantly lower than HBM or even DRAM-based memory expansions, which could be a major advantage for hyperscalers building new data centers in the current AI boom.
Chipmakers such as SanDisk and SK Hynix are attempting to overcome AI's memory wall by developing the novel High Bandwidth Flash memory solution. Meanwhile, Kioxia continues to invest heavily in traditional, albeit massively capable, storage formats such as its new 245TB LC9 enterprise SSDs.
Earlier this year, the Japanese company said that affordable consumer SSDs costing $50 per terabyte are now a thing of the past. However, the manufacturer is still working to bring solid-state storage solutions to budget-friendly PC builds with its recently introduced EG7 Series, which is based on quadruple-level cell (QLC) NAND technology.