CODEWITHSUNDEEP
pythonpython-3.xcorrelationpython-xarraypearson
kadau92
kadau92

Reputation: 1

Pearson correlation matrix in Python, ValueError

I'm trying to get a matrix, (lat, lon) size, with the Pearson Coefficient value for every grid point, for

x : a 3D DataArray (time, lat, lon) (time size is 30)

y : a DataArray column vector with a 30 values series inside

So i would like to calculate the pearson coefficient for every (lat,lon) for a column vector of 30 elements for x.

I tried:

corrmap = xr.DataArray(x2)
for i in range(len(corrmap['lat']))
     for j in range(len(corrmap['lon']))
          corrmap[i, j], p_value = pearsonr(x[:, i, j], y)

but i get this error:

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

that i cannot perfectly understand in the meaning. Is my method uncorrect? Should i use another type of code to solve my problem?

Any help would be greatly appreciated.

Upvotes: 0

Views: 2191

Answers (3)

Bendy Latortue
Bendy Latortue

Reputation: 401

You can calutate the Pearson correlation coefficient using:

import numpy
numpy.corrcoef(list1, list2)[0, 1]

Upvotes: 0

JulianGiles
JulianGiles

Reputation: 338

The problem is you are using y and it has two dimensions, pearsonr can't handle that. Specify y[:,0] and it works. That is:

corrmap = np.zeros(((len(corrmap['lat']), len(corrmap['lon'])))

for i in range(len(corrmap['lat'])):
     for j in range(len(corrmap['lon'])):
          corrmap[i, j], p_value = pearsonr(x[:, i, j], y[:,0])

Also I would just use a numpy array for the coefficients instead of an xarray, at least to get the values, then you can convert it to xarray.

Upvotes: 2

Konstantin Grigorov
Konstantin Grigorov

Reputation: 1602

Assuming that you are using scipy.stats.pearsonr. The arguments to PearsonR should be one-dimensional arrays.

So give a go to:

corrmap[i, j], p_value = pearsonr(x[:, i, j].ravel(), y)

What ravel() does is return contiguous flattened array, Numpy ravel()

Here is also a bit of context around the error that you are getting, evaluate array in boolean context. In other words, the extra dimension is probably causing some operations to be applied to an array instead of a scalar. This issue is of the same nature as yours: SO: PearsonR ValueError

Upvotes: 0

Related Questions