CODEWITHSUNDEEP
chdf5
Mauro Ganswer
Mauro Ganswer

Reputation: 1419

HDF5: How to append data to a dataset (extensible array)

By following this tutorial, I've tried to extend my HDF5 dataset. The code is the following, however the data is not properly written to the dataset (the dataset has the proper final size but contains only zeros). The only difference from the tutorial is that I have to use dynamic arrays. Any idea?

int main()
{    
    hsize_t dims[1], max_dims[1], newdims[1], chunk_dims[1], offset[1];
    hid_t file, file_space, plist, dataset, mem_space;
    int32_t *buffer1, *buffer2;

    file = H5Fcreate("test.h5", H5F_ACC_TRUNC, H5P_DEFAULT, H5P_DEFAULT);

    // Create dataspace with initial dim = 0 and final = UNLIMITED
    dims[0] = 0;
    max_dims[0] = H5S_UNLIMITED;
    file_space = H5Screate_simple(RANK, dims, max_dims);

    // Create dataset creation property list to have chunks
    plist = H5Pcreate(H5P_DATASET_CREATE);
    H5Pset_layout(plist, H5D_CHUNKED);
    chunk_dims[0] = 2;
    H5Pset_chunk(plist, RANK, chunk_dims);

    // Create the dataset
    dataset = H5Dcreate(file, "Test", H5T_NATIVE_INT32, file_space, H5P_DEFAULT, plist, H5P_DEFAULT);

    H5Pclose(plist);
    H5Sclose(file_space);

    //## FIRST BUFFER

    int length = 6;
    buffer1 = new int32_t[length];
    for (hsize_t i = 0; i < length; i++)
        buffer1[i] = i;

    // Extend the dataset by getting previous size and adding current length
    file_space = H5Dget_space(dataset);
    H5Sget_simple_extent_dims(file_space, dims, NULL);
    newdims[0] = dims[0] + length;
    H5Dset_extent(dataset, newdims);

    // Select hyperslab on the file dataset
    offset[0] = dims[0];
    dims[0] = length;
    H5Sselect_hyperslab(file_space, H5S_SELECT_SET, offset, NULL, dims, NULL);

    // Dataspace for buffer in memory
    mem_space = H5Screate_simple(RANK, dims, NULL);

    // Append buffer to dataset
    H5Dwrite(dataset, H5T_NATIVE_INT32, mem_space, file_space, H5P_DEFAULT, buffer1);

    H5Sclose(file_space);
    H5Sclose(mem_space);

    //## SECOND BUFFER

    length = 4;
    buffer2 = new int32_t[length];
    for (hsize_t i = 0; i < length; i++)
        buffer2[i] = i;

    // Extend the dataset by getting previous size and adding current length
    file_space = H5Dget_space(dataset);
    H5Sget_simple_extent_dims(file_space, dims, NULL);
    newdims[0] = dims[0] + length;
    H5Dset_extent(dataset, newdims);

    // Select hyperslab on the file dataset
    offset[0] = dims[0];
    dims[0] = length;
    H5Sselect_hyperslab(file_space, H5S_SELECT_SET, offset, NULL, dims, NULL);

    // Dataspace for buffer in memory
    mem_space = H5Screate_simple(RANK, dims, NULL);

    // Append buffer to dataset
    H5Dwrite(dataset, H5T_NATIVE_INT32, mem_space, file_space, H5P_DEFAULT, buffer2);

    H5Sclose(file_space);
    H5Sclose(mem_space);

    H5Dclose(dataset);
    H5Fclose(file);

   delete[] buffer1;
   delete[] buffer2;
}

Upvotes: 3

Views: 3473

Answers (1)

Mauro Ganswer
Mauro Ganswer

Reputation: 1419

I've figured out how to solve the problem. It has nothing to do with the dynamic arrays. The problem is that after calling H5Sget_simple_extent_dims the dataspace id gets somehow invalidated (I do not understand why...a bug?) and you need to get it again before reusing it, e.g. before selecting the hyperslab:

// Select hyperslab on the file dataset
offset[0] = dims[0];
dims[0] = length;
H5Dclose(file_space); // --ADDED-- CLOSE THE PREVIOUSLY OPENED
file_space = H5Dget_space(dataset); // --ADDED-- REOPEN
H5Sselect_hyperslab(file_space, H5S_SELECT_SET, offset, NULL, dims, NULL);

Upvotes: 3

Related Questions