CODEWITHSUNDEEP
pythonhdf5h5py
lr100
lr100

Reputation: 598

Updating one field in HDF File

I can't seem to get this to work. All the examples and threads all have people creating new datasets. I just want to update one field in a dataset that is already created.

Here is what I have:

h5_file = h5py.File(event_file_path, "r+") #this works

event_processing_status = int(h5_file[PATH][STATUS].value[0]['Status']) #this works
print({"{0:b}".format(event_processing_status)) #this works
event_processing_status = (event_processing_status | STATUS_UPDATE) #this works
h5_file[PATH][STATUS].value[0]['Status'] = event_processing_status #updating???, no error
event_processing_status = int(h5_file[PATH][STATUS].value[0]['Status']) #this works
print({"{0:b}".format(event_processing_status)) #not the update value

h5_file.close()

What am I doing wrong?

More info: The dtypes of the columns of the dataset:

dset = h5_file[PATH][STATUS] 
print(dset.dtype) gives:
[('Status', '<u8'), ('Segments', '<u4'), ('Characterized', '<u4'), ('More_Segments', '<u4'), ('ID', '<i4'), ('Releases', '<u2'), ('Type', '|u1'), ('Track', '<i4')]

dset[0,'Status'] = event_processing_status gives:
TypeError: Field name selections are not allowed for write.

Upvotes: 2

Views: 1849

Answers (2)

lr100
lr100

Reputation: 598

This is what I had to do:

h5_file = h5py.File(event_file_path, "r+") #this works

#Retrieve the dataset containing Event_Status
data= h5_file[PATH][STATUS]

#Get the Event_Status (np.array)
event_status = data['Event_Status']

#Update the value of Event_Status
event_status[0] = np.bitwise_or(event_status[0],np.uint64(STATUS_UPDATE))

#Write updated value to file
elements = data[0]
elements['Event_Status'] = event_status
data[0] = elements

h5_file.close()

Upvotes: 1

hpaulj
hpaulj

Reputation: 231738

Following up on my comment, assuming that your dataset is a structured/compound dtype

In [144]: f = h5py.File('test.h5','w')
In [145]: arr = np.ones((3,), dtype='i,f')  # structured array
In [146]: arr
Out[146]: 
array([(1,  1.), (1,  1.), (1,  1.)],
      dtype=[('f0', '<i4'), ('f1', '<f4')])

create dataset with data

In [147]: ds = f.create_dataset('arr',data=arr)
In [148]: ds
Out[148]: <HDF5 dataset "arr": shape (3,), type "|V8">
In [149]: ds.value
Out[149]: 
array([(1,  1.), (1,  1.), (1,  1.)],
      dtype=[('f0', '<i4'), ('f1', '<f4')])

I can index it with record id and field name; this doesn't work for ds.value or for arr.

In [151]: ds[0,'f0']
Out[151]: 1
In [152]: ds[0,'f0'] = 2    # and I can assign values
In [153]: ds.value
Out[153]: 
array([(2,  1.), (1,  1.), (1,  1.)],
      dtype=[('f0', '<i4'), ('f1', '<f4')])

I can index with separate record and field entries; but can't change values this way:

In [154]: ds[0]['f1']
Out[154]: 1.0
In [155]: ds[0]['f1'] = 234
In [156]: ds.value
Out[156]: 
array([(2,  1.), (1,  1.), (1,  1.)],
      dtype=[('f0', '<i4'), ('f1', '<f4')])
In [157]: ds['f1'][0] = 234
In [158]: ds.value
Out[158]: 
array([(2,  1.), (1,  1.), (1,  1.)],
      dtype=[('f0', '<i4'), ('f1', '<f4')])

Assignment requires the combined indexing

In [159]: ds[0,'f1'] = 234
In [160]: ds.value
Out[160]: 
array([(2,  234.), (1,    1.), (1,    1.)],
      dtype=[('f0', '<i4'), ('f1', '<f4')])

Upvotes: 1

Related Questions