CODEWITHSUNDEEP
pythonnumpymatrixtensorflow
Garland
Garland

Reputation: 65

Positive semi-definite error when sampling from multivariate norm in python?

NB: this problem actually happens inside tensorflow, and results in samples not exactly from the true pdf. The principle is, however, the same in numpy, and my aim is to understand the following warning.

Namely, I am trying to sample from a multivariate normal in python. That is

np.random.multivariate_normal(mean = some_mean_vector, cov = some_cov_matrix)

Of course, any valid covariance matrix must be positive semi-definite. However, some covariance matrices used for sampling (that pass every test for positive semi-definiteness), give the following warning

/usr/local/lib/python3.6/site-packages/ipykernel/__main__.py:1: RuntimeWarning: covariance is not positive-semidefinite.

One such matrix is

A = array([[  1.00000359e-01,  -3.66802835e+00],[ -3.66802859e+00,   1.34643845e+02]], dtype=float32)

for which I can find both the cholesky decomposition and eigenvalues without warning (the smallest eigenvalue is 7.42144039e-05).

Can anyone help and tell me why this might be happening?

(In tensorflow, I just feed the cholesky decomposition of the above matrix, and receive inexact samples, which messes up everything I'm trying to do).

Upvotes: 0

Views: 1835

Answers (1)

user8153
user8153

Reputation: 4095

The discussion of this pull request has some information about what triggers this warning. According to the source:

   # Also check that cov is positive-semidefinite. If so, the u.T and v
   # matrices should be equal up to roundoff error if cov is
   # symmetrical and the singular value of the corresponding row is
   # not zero. We continue to use the SVD rather than Cholesky in
   # order to preserve current outputs. Note that symmetry has not
   # been checked.

It seems that in your case the tests based on SVD and Cholesky give different results.

Upvotes: 1

Related Questions