CODEWITHSUNDEEP
pythonnumpy
piccolo
piccolo

Reputation: 2227

Python merging npz files

Is there anyway to merge npz files in python. In my directory I have output1.npz and output2.npz.

I want a new npz file that merges the arrays from both npz files.

Upvotes: 5

Views: 5034

Answers (3)

Pau Gonzalez
Pau Gonzalez

Reputation: 61

You have surely solved the problem by now (1 year and 10 months later...), but I just came across the same problem and found a solution that might be worth sharing here.

In general, if you have a list of .npz files file_list = ['file_0.npz', 'file_1.npz', ...], eventually also with particular naming, i.e. the file was created using **kwargs and not simply *args, one could do the following:

import numpy as np

data_all = [np.load(fname) for fname in file_list]
merged_data = {}
for data in data_all:
    [merged_data.update({k: v}) for k, v in data.items()]
np.savez('new_file.npz', **merged_data)

I was using python 3.7.7 with numpy 1.18.1.

Cheers!

Upvotes: 0

Swike
Swike

Reputation: 192

If you have 3 npz files ('Data_chunk1.npz', 'Data_chunk2.npz' and 'Data_chunk3.npz'), all containing the same number of arrays (in my case 7 different arrays), then you can do

import numpy as np

# Load the 3 files
data_1 = np.load('Data_chunk1.npz')
data_2 = np.load('Data_chunk2.npz')
data_3 = np.load('Data_chunk3.npz')

# Merge each of the 7 arrays of the 3 files
arr_0 = np.concatenate([data_1['arr_0'], data_2['arr_0'], data_3['arr_0']])
arr_1 = np.concatenate([data_1['arr_1'], data_2['arr_1'], data_3['arr_1']])
arr_2 = np.concatenate([data_1['arr_2'], data_2['arr_2'], data_3['arr_2']])
arr_3 = np.concatenate([data_1['arr_3'], data_2['arr_3'], data_3['arr_3']])
arr_4 = np.concatenate([data_1['arr_4'], data_2['arr_4'], data_3['arr_4']])
arr_5 = np.concatenate([data_1['arr_5'], data_2['arr_5'], data_3['arr_5']])
arr_6 = np.concatenate([data_1['arr_6'], data_2['arr_6'], data_3['arr_6']])

# Save the new npz file
np.savez('Data_new.npz', arr_0, arr_1, arr_2, arr_3, arr_4, arr_5, arr_6 )

Upvotes: 0

Jiafeng Liao
Jiafeng Liao

Reputation: 49

use numpy.load('output1.npz') and numpy.load('output2.npz') to load both files as array a1,a2. Then use a3 =[*a1,*a2] to merge them. Finally, output via numpy.savez('output.npz',a3)

Upvotes: 4

Related Questions