CODEWITHSUNDEEP
c#
Igor D
Igor D

Reputation: 19

Working with large array on disk

I need to work with a large 2-dimensional array of doubles, with more than 100 million cells. The matrix first needs to be filled and then manipulated by taking either one row or one column. The matrix can be bigger than 1 terabyte in size, and will not find in memory.

How can the array be stored efficiently? The main operation is fast saving it from memory row by row (double[100k] each) and fast reading to the memory of one row or one column.

Upvotes: 1

Views: 410

Answers (1)

Nican
Nican

Reputation: 7945

You could use Memory Mapped Files. You are essentially still working with an array, but allowing the kernel to choose what parts to load into memory. You could also possibly use Fixed size buffers to read whole sections of the memory mapped files.

Upvotes: 3

Related Questions