DeepSeek open-sources file system, claims it runs AI models faster and more efficiently

[Alfonso Maruccia]

What just happened? In response to Western organizations calling it "shady and untrustworthy," DeepSeek launched "Open Source Week." During last week's event, the company released several repositories to the open-source community, including a highly efficient file system. Many AI specialists reviewing the code have come away impressed.

Last week, DeepSeek released five of its most advanced software repositories during its "Open Source Week" event. The Chinese AI firm unveiled a Linux-based file system it uses internally for AI training and inference workloads. The Fire-Flyer File System (3FS) boasts some impressive performance benchmarks. Western AI companies have taken note and are exploring the repos. The company designed 3FS to accelerate AI tasks. The technology leverages the features of modern solid-state storage units and RDMA networks, providing a shared storage layer to simplify the deployment of distributed applications.

Tom's Harware notes that DeepSeek's 3FS code works without read caching and prioritizes random read requests since AI models running on GPU nodes constantly access data snippets stored on servers. The file system can combine the throughput of thousands of SSD units and the network bandwidth of hundreds of storage nodes, simplifying application code and applying standard storage API models.

🚀 Day 5 of #OpenSourceWeek: 3FS, Thruster for All DeepSeek Data Access

Fire-Flyer File System (3FS) - a parallel file system that utilizes the full bandwidth of modern SSDs and RDMA networks.

⚡ 6.6 TiB/s aggregate read throughput in a 180-node cluster
⚡ 3.66 TiB/min…

– DeepSeek (@deepseek_ai) February 28, 2025

The distributed file system can reach a 6.6 TiB/s aggregate read throughput when used in a 180-node cluster, achieving a 3.66 TiB/min throughput on the GraySort benchmark (in a 25-node cluster). Startup company Perspective AI praised DeepSeek's figures as some "next-level" benchmarks, describing 3FS as a potential revolution for data-heavy workloads related to AI, research, and more.

In a paper published last summer, DeepSeek researchers described the features of the company's custom Fire-Flyer 2 AI high-performance computing architecture. Thanks to 3FS, HaiScale, and other elements of its software stack, DeepSeek achieved 80 percent of the performance of Nvidia's DGX-A100 servers at 50 percent of the price and using 40 percent less energy. Fire-Flyer 2 used 180 storage nodes with 16 16TB SSDs each, two 200Gbps NUCs, and 10,000 Nvidia A100 GPUs over PCIe.

DeepSeek created Open Source Week to emphasize its transparency and community-based innovation after being criticized as shadowy and untrustworthy. The Chinese company is releasing many software products as open-source repositories, with key targets including FlashMLA, DeepEP, DeepGEMM, and more.

Permalink to story:

DeepSeek open-sources file system, claims it runs AI models faster and more efficiently

 

[The Talking Tech]

It's a great model, the fact that it's Chinese it doesn't mean is inferior in anyway, it only means the U.S entities are over spending on theirs in comparison.

 

[mosu]

Impressive!

 

[Kirby1]

It's a great model, the fact that it's Chinese it doesn't means is inferior in anyway, it only means the U.S entities are over spending on theirs in comparison.

There have been and will continue to be many advances in AI both in hardware of software. When it comes to the software, there will be tons of innovation in efficiency, things are still pretty early on for AI. It's news when China is able to innovate like this because it's rare for China to have impressive innovations on major tech in general.

 

[GodisanAtheist]

Between Deepseek and the Trump Tariffs it looks like the US is going to speed run a recession/depression.

 

[daffy duck]

More shady than Altman, Fcukerberg or Musk?

 

[Liranan]

Having tested DS 14 and 32B I can say that it's far better than Llama, Snoozy and others, it actually gives useful answers, unlike the nonsense that the others spew out.