CODEWITHSUNDEEP
c++carrays
andrw
andrw

Reputation: 810

Huge 2D arrays in C++

I'm working on some code that does image manipulation, and it reads every 16 bits from a raw image file and stores each block into an array. The array needs 1392 columns and 1024 rows. I'm reading in data blocks from the raw file with fread:

    fread(&q1[0][0], sizeof(uint16_t), NUM_COLS*NUM_ROWS*sizeof(uint16_t), fp);

which seems to work up until q1[0][280], where it suddenly stops (values past 280 are 0).

I had previously declared arrays directly:

uint16_t q1[NUM_COLS][NUM_ROWS];

but I thought that it would need dynamic allocation to store more than 280 values, so I re-wrote it to be 

    uint16_t** arr;
    arr= (uint16_t**) malloc(NUM_ROWS * sizeof(uint16_t *));
if (arr == NULL) {
    fprintf(stderr, "out of memory\n");

}
for(int i=0; i<NUM_ROWS; i++) {
    arr[i]= (uint16_t*) malloc(NUM_COLS * sizeof(uint16_t));
    if(arr[i] == NULL) {
        fprintf(stderr, "out of memory\n");

    }
}

Unfortunately, this still stops at index 280. Any ideas as to why it's stopping (or if there's a better way of doing this)?

Upvotes: 2

Views: 512

Answers (5)

Dan
Dan

Reputation: 12675

If you're making your array dynamic, you might want to consider:

or

  • Allocating a one-dimensional array: 

    arr= (uint16_t *) malloc(NUM_ROWS*NUM_COLS*sizeof(uint16_t));

    and accessing it at column col and row row as follows:

    arr[col*NUM_ROWS+row]

    This is not as convienient to use, but its simple to implement.

Upvotes: 0

Potatoswatter
Potatoswatter

Reputation: 137810

First, it's better not to presume that such an array will be too big. Most likely your platform does support a global array of a couple megabytes. Really you're not talking about a very big object at all.

Second, an array of pointers is most useful when the row size varies within the array of arrays. For a graphics application, this will degrade spatial locality and hurt performance. Also, you will call malloc and free thousands of times more than necessary, which can also add up.

The problem with reading NUM_COLS*NUM_ROWS*sizeof(uint16_t) bytes at once is that the image is no longer contiguous in memory; it's divided into a separate block of memory for each row. Try a separate I/O operation for each malloced block… although my recommendation would be to reconsolidate that block instead, and if its size is really constant, make it a global.

Upvotes: 5

Timbo
Timbo

Reputation: 28050

Your two-dimensional array is an array of arrays.

q1[0] is the address of first subarray, q1[0][0] is the first element in the first subarray. However, the second subarray q1[1][0] is the result of a different call to malloc. Therefore, it is not in consecutive memory after the first subarray.

Your read call fills the first row, and then should cause a segmentation fault afterwards, because the memory is not what you expect it to be.

You will have to read the data in steps, one for each subarray.

Upvotes: 4

Bo Persson
Bo Persson

Reputation: 92271

If this is your actual code for reading

fread(&q1[0][0], sizeof(uint16_t), NUM_COLS*NUM_ROWS*sizeof(uint16_t), fp);

you attempt to read way more than fits in the array. The second parameter is the size of each element and the third is number of elements. You cannot have sizeof in both.

Did you check the return value of fread, the number of elements actually read? If that is larger than the array, all bets are off!

When you later change to allocating each row separately, you will also have to read each row separately, because the whole matrix is no longer in contiguous memory.

Upvotes: 2

Andrew Stein
Andrew Stein

Reputation: 13170

You are reading NUM_COLS*NUM_ROWS integers into a location (&q1[0][0]) which is not the start of an allocated block of NUM_COLS*NUM_ROWS integers.

What you should do is:

for (int i = 0; i < NUM_ROS; ++i) {
    fread (q[i], sizeof(uint16_t), NUM_COLS, fp);
}

The reason for this is that, according to the way you allocated, each 'row' is the start of an allocated block of NUM_COLS integers.

Also, not that the 3rd argument to fread is the number of items, not the size of the items.

Upvotes: 2

Related Questions