Intel’s DAOS tops the IO500 speed challenge for HPC storage
Summary
Intel wants DAOS to be a successor to the Lustre file system, which it previously held rights to but sold to DDN in 2018. On the compute side, the DAOS file system handled requests via Posix, HDFS, Apache Spark and MPI I/O (an HPC-specific parallel I/O scheme). To maximise access speed, DAOS doesn’t provide any mechanism for replication or erasure coding for data that is already saved. According to industry observers, DAOS is perhaps the heaviest consumer of hardware resources – and therefore the most expensive – of all cluster-based file storage solutions. HPC used to mean mostly large batch processed calculation workloads done on mainframes and supercomputers, often in scientific work, weather prediction, oil and gas prospecting, and so on.