CODEWITHSUNDEEP
pythonpython-2.7hdf5h5py
John
John

Reputation: 3070

How to compress the data that saved in hdf5?

I am using python 2.7 to read a video and store in hdf5. This is my code

import h5py
import skvideo.datasets
import skvideo.io
videodata = skvideo.io.vread('./v_ApplyEyeMakeup_g01_c01.avi')
with h5py.File('./video.hdf5','w') as f:
    f['data'] = videodata
    f['label'] = 1

The problem is that the output hdf5 is too larger. It is 128 times larger than the original avi file. What should I do to compress/reduce the size? You can download the file at https://drive.google.com/open?id=0B1MrjZsURl2yNFM0ZTJfZ3pOZVU

I think we can compress it by using

f.create_dataset('data',data=videodata,compression='gzip',compression_opts=9)
f.create_dataset('label', data=1)

Now, it still 37 times larger than the original file. Thanks in advance.

Upvotes: 0

Views: 11565

Answers (3)

Bhargav Reddy
Bhargav Reddy

Reputation: 1

After saving the model into HDF5, you need to load the model and save the weights of it. By this H5 or HDF5 file size will be reduced.

Upvotes: -1

kakk11
kakk11

Reputation: 918

By adding chunking I was able to make the output 7.2M compared to 10M without. So it definitely improves, but still far from dedicated video formats. You may play with other filters from https://support.hdfgroup.org/services/filters.html but I doubt they will improve the compression by an order of magnitude. So if you want to continue with h5py, you probably need to accept larger file size. In case this is not acceptable, just try another file format.

import h5py
import skvideo.datasets
import skvideo.io
videodata = skvideo.io.vread('./v_ApplyEyeMakeup_g01_c01.avi')

print(videodata.shape)
with h5py.File('./video.hdf5','w') as f:
    f.create_dataset('data',
                      data=videodata,
                      compression='gzip',
                      compression_opts=9,
                      chunks=(164, 20, 20, 3))
    f.create_dataset('label', data=1)

Upvotes: 1

Sinapse
Sinapse

Reputation: 792

Your problem should be solved using a suitable encode for your video file. Based on your business, there are various encoding algorithms for example there is x265 which will compress the video but requires high resource to do that. Take a look here.

Recently I have heard about another interesting encode which is good for online streaming called Daala you can get more information here.

Generally it depends on what you expect from the encoding, but choosing a good encoder is the way you should go, try search for that.

Upvotes: 1

Related Questions