CODEWITHSUNDEEP
pythonh5py
Alex
Alex

Reputation: 961

h5py clearing file consistency flags

I did some research and I stored the results in an HDF5 file using the h5py module. I opened and read the data a bunch of times using both the h5py module and the HDF view tool from the HDF5 group. This all worked fine, until one day my computer crashed while the file was open in HDF view.

After rebooting the pc I could no longer open the file. The HDF view tool shows a generic error: "Error opening file "

I wrote the file in h5py, so I decided to try and use this for reading the data aswell. The file was written in swmr mode with libver='latest'. I tried the following:

with h5py.File(fpath, 'r', swmr=True, libver='latest') as f:
    pass

Returns an error "OSError: Unable to open file (file is not already open for SWMR writing)"

with h5py.File(fpath, 'r') as f:
    pass

Returns an error "OSError: Unable to open file (file is already open for write (may use h5clear file to clear file consistency flags))"

Now I'm wondering, is the h5clear option implemented in the h5py module yet? I cannot find any information about this anywhere.

Edit: Removed the file (sorry)

Upvotes: 6

Views: 5039

Answers (1)

Steven C. Howell
Steven C. Howell

Reputation: 18654

Given a hdf5 file that throws this error, Unable to open file (file is already open for write/SWMR write), where you do not have a way to recreate the file, you can clear the file consistency flag using the command line tool h5clear. 

$> h5clear -s my_bad.h5

One way to get the h5clear utility (on Windows 10 or any other OS) is by installing h5py (or pandas, not sure which was responsible) using the Anaconda Python distribution. On my system, the executable was located in the environment bin directory: anaconda3/envs/my_env/Library/bin/h5clear. I expect you can also get this utility by installing h5py from pip though I have not tested this.

If you have Anaconda installed, you can create an environment, install the packages, then run h5clear using the following commands from the command line. On Windows, I use git-bash but this should also work from the Anaconda Prompt, or even the Windows Command Prompt if you setup your path correctly.

$> conda create --name demo

$> source activate demo

(demo) 
$> conda install h5py pandas

(demo)
$> h5clear -s my_bad.h5

Upvotes: 3

Related Questions