CODEWITHSUNDEEP
pythonh5py
Somnath Jana
Somnath Jana

Reputation: 53

ValueError: Unable to create group (name already exists) in h5py

I am trying to save some data in h5py format. I would like to rewrite an existing file. Below is an example code.

import h5py
hf = h5py.File('fileName.h5py', 'w')
grp = hf.create_group('grp_1')
data = np.random.randn(100)
dataset = grp.create_dataset(file, data=data)

Problems: If there is already a fileName.h5py file, it complains:

'OSError: Unable to create the file (unable to truncate a file which is >already open)'.

If there is already a group 'grp_1', it complains:

'ValueError: Unable to create group (name already exists) in h5py'.

But according to the documentation, these errors should not appear or??

Any help will be appreciated. I am using the 2.10.0 version of h5py and python 3.8.8.

Upvotes: 1

Views: 4491

Answers (1)

kcw78
kcw78

Reputation: 8046

You have 2 errors.
First error: OSError: Unable to create file (unable to truncate a file which is already open)
This error occurs when you try to open with when another process has the file open in write mode ('w'). Only 1 process can open a HDF5 file for write access. In the code segment above you don't have hf.close(). Thus it is still open. Open files also occur when you have a exception/error when running and exit without getting to the close statement (so don't close the file properly).

To avoid this situation, use Python's context manager. It will automatically close the file when you exit the with/as code block (even when there is an exception) like this:

import h5py
with h5py.File('fileName.h5py', 'w') as hf:
    grp = hf.create_group('grp_1')
    data = np.random.randn(100)
    dataset = grp.create_dataset(file, data=data)

Second error: ValueError: Unable to create group (name already exists) when there is already a group 'grp_1'. This is intended behavior. It avoids unintentionally overwriting an existing group (and losing existing data).

Solution: create_group() has an alternate function you can use when you don't know if it exists. It is: require_group(). If the group doesn't exist, it will create it. If it exists, it will return the group object. There is a similar pair for create_dataset() and require_dataset().

Upvotes: 4

Related Questions