CODEWITHSUNDEEP
linuxfilesystemsstoragenvmeraid
shaiko
shaiko

Reputation: 179

Striping data: true RAID 0 vs user program

I want to build a high performance recording device that receives streaming data from a PCIe 5.0 card and stores it into a storage array. The throughput is 10 Gbytes/s and I need to record 20 TBytes without loosing a single bit.

To meet the high throughput and volume requirement my idea is to stripe the data across multiple SSDs such that the PCIe data will be divided into sections and written simultaneously to all SSDs. These are my ideas for the implementation :

  1. Use a RAID 0 utility such as mdadm, segment the incoming data into files and use the native write() function of the OS to write the data to the single virtual device.
  2. Striping handled by the user's program. The same segment from #1 will be split into a number of files (equal to the number of physical SSD device) with all files written simultaneously to its matching SSD. The idea is to leverage the dataflow simplicity of my application (non random access, constant dataflow, etc...).

Do you expect a significant performance boost with the 2nd approach ? Any other suggestions ?

Upvotes: 0

Views: 64

Answers (0)

Related Questions