CODEWITHSUNDEEP
pythonarraysnumpyhistogram
Peter Prettenhofer
Peter Prettenhofer

Reputation: 1971

Numpy histogram on multi-dimensional array

given an np.array of shape (n_days, n_lat, n_lon), I'd like to compute a histogram with fixed bins for each lat-lon cell (ie the distribution of daily values).

A simple solution to the problem is to loop over the cells and invoke np.histogram for each cell::

bins = np.linspace(0, 1.0, 10)
B = np.rand(n_days, n_lat, n_lon)
H = np.zeros((n_bins, n_lat, n_lon), dtype=np.int32)
for lat in range(n_lat):
    for lon in range(n_lon):
        H[:, lat, lon] = np.histogram(A[:, lat, lon], bins=bins)[0]
# note: code not tested

but this is quite slow. Is there a more efficient solution that does not involve a loop?

I looked into np.searchsorted to get the bin indices for each value in B and then use fancy indexing to update H::

bin_indices = bins.searchsorted(B)
H[bin_indices.ravel(), idx[0], idx[1]] += 1  # where idx is a index grid given by np.indices
# note: code not tested

but this does not work because the in-place add operator (+=) doesn't seem to support multiple updates of the same cell.

thx, Peter

Upvotes: 7

Views: 4399

Answers (2)

Greg Whittier
Greg Whittier

Reputation: 3185

You can use numpy.apply_along_axis() to eliminate the loop.

import numpy as np

hist, bin_edges = np.apply_along_axis(lambda x: np.histogram(x, bins=bins), 0, B)

Upvotes: 4

Raphael Roth
Raphael Roth

Reputation: 27373

Maybe this works?: 

import numpy as np
n_days=31
n_lat=10
n_lon=10
n_bins=10
bins = np.linspace(0, 1.0, n_bins)
B = np.random.rand(n_days, n_lat, n_lon)


# flatten to 1D
C=np.reshape(B,n_days*n_lat*n_lon)
# use digitize to get the index of the bin to which the numbers belong
D=np.digitize(C,bins)-1
# reshape the results back to the original shape
result=np.reshape(D,(n_days, n_lat, n_lon))

Upvotes: 0

Related Questions