CODEWITHSUNDEEP
c++hdf5
steveo225
steveo225

Reputation: 11882

Writing a 3D array to HDF5

I am trying to write a 3D data set to an HDF5 file. The write succeeds and the dimensions are all correct in the file, however, when using the HDFView tool, the data is jumbled, but has a recognizable structure, so I must be close. I can't figure out what I am missing and can't find a good example for writing 3D data.

The data is a contiguous chunk of memory indexed as such: data[layer][row][col] or data[layer*rows*cols + row*cols + col]

The following is the code I am using, where HDF5_CHECK is a macro that throws an exception on error.

const hsize_t dim[] = { rows, cols, layers };
hid_t space = H5Screate_simple(3, dim, NULL);

hid_t dataset = H5Dcreate(file, "dataset", type, space, H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);
HDF5_CHECK(dataset);
HDF5_CHECK(H5Dwrite(dataset, type, H5S_ALL, H5S_ALL, H5P_DEFAULT, data));

HDF5_CHECK(H5Sclose(space));
HDF5_CHECK(H5Dclose(dataset));

EDIT

I believe the issue is related to indexing. The array in C++ is correct. However, I believe I want to store the Z, Y, X data as Y, X, Z in the HDF5 container. This must be possible using hyperslabs, but I am unsure how

EDIT 2

Since the data has structure, I can view the data as an image in HDFView. The following should help show the issue:

EDIT 3

Contacted the HDF group and found out shifting dimensions is not possible in the current implementation. It must be done before writing the dataset.

Expected vs Actual results

Upvotes: 3

Views: 2937

Answers (1)

E_net4
E_net4

Reputation: 30013

The elements in the dataset are not in the expected order because an incorrect memory dataspace was specified.

The user guide in section 4 shows how HDF5 can read between two dataspaces of the same overall size, but with swapped dimensions from file to memory. In practice, the H5Dwrite function is declared like this:

herr_t H5Dwrite(hid_t dataset_id, hid_t mem_type_id, hid_t mem_space_id, hid_t file_space_id, hid_t xfer_plist_id, const void * buf )

The third parameter refers to the memory dataspace, which in our case is not the same as the file's (previously defined as (row, col, layer)). It should instead be defined as (layer, row, col). The same page in the guide clarifies that HDF5 follows the same array storage convention as in C:

Dataspace dimensions are numbered from 1 to rank. HDF5 uses C storage conventions, assuming that the last listed dimension is the fastest-changing dimension and the first-listed dimension is the slowest changing. The HDF5 file format storage layout specification adheres to the C convention and the HDF5 Library adheres to the same convention when storing dataspace dimensions in the file.

The corrected code should be something like this:

const hsize_t dim[] = { rows, cols, layers };
hid_t space = H5Screate_simple(3, dim, NULL);

hid_t dataset = H5Dcreate(file, "dataset", type, space, H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);

HDF5_CHECK(dataset);

const hsize_t mem_dim[] = { layers, rows, cols };
hid_t mem_space = H5Screate_simple(3, dim, NULL);
HDF5_CHECK(H5Dwrite(dataset, type, mem_space, H5S_ALL, H5P_DEFAULT, data));

Upvotes: 1

Related Questions