CODEWITHSUNDEEP
c++hdf5armadillo
laolux
laolux

Reputation: 1565

Armadillo: Save multiple datasets in one hdf5 file

I am trying to save multiple datasets into a single hdf5 file using armadillo's new feature to give custom names to datasets (using armadillo version 8.100.1). However, only the last saved dataset will end up in the file. Is there any way to append to an existing hdf5 file with armadillo instead of replacing it?

Here is my example code:

#define ARMA_USE_HDF5
#include <armadillo>

int main(){
    arma::mat A(2,2, arma::fill::randu);
    arma::mat B(3,3, arma::fill::eye);
    A.save(arma::hdf5_name("multi-hdf5.mat", "dataset1"), arma::hdf5_binary);
    B.save(arma::hdf5_name("multi-hdf5.mat", "dataset2"), arma::hdf5_binary);
    return 0;
}

The hdf5 file is read out using the h5dump utility.

Upvotes: 1

Views: 1005

Answers (2)

Zythos
Zythos

Reputation: 180

The latest version of Armadillo already covers that possibility. You have to use hdf5_opts::append in the save method so if you want to save a matrix A then you can write A.save(hdf5_name(filename, dataset, hdf5_opts::append) ).

Upvotes: 1

Dana Robinson
Dana Robinson

Reputation: 4364

Unfortunately, I don't think you can do that. I'm an HDF5 developer, not an armadillo developer, but I took a peek at their source for you.

The save functions look like they are designed to dump a single matrix to a single file. In the function save_hdf5_binary() (diskio_meat.hpp:1255 for one version) they call H5Fcreate() with the H5F_ACC_TRUNC flag, which will clobber any existing file. There's no 'open if file exists' or clobber/non-clobber option. The only H5Fopen() calls are in the hdf5_binary_load() functions and those don't keep the file open for later writing.

This clobbering is what is happening in your case, btw. A.save() creates a file containing dataset1, then B.save() clobbers that file with a new file containing dataset2.

Also, for what it's worth, 'appending to an HDF5 file' is not really the right way to think about that. HDF5 files are not byte/character streams like a text file. Appending to a dataset, yes. Files, no. Think of it like a relational database: You might append data to a table, but you probably wouldn't say that you were appending data to the database.

Upvotes: 1

Related Questions