CODEWITHSUNDEEP
python-3.xnumpyscipy
16Aghnar
16Aghnar

Reputation: 113

Scipy and Numpy upgrade generates "TypeError: Cannot cast array data from dtype('O') to dtype('float64')"

I'm converting a code from Python 2.7 to Python 3.8.

In its Python 2.7 version, I had to use downgraded versions of scipy and numpy in order to avoid a TypeError (see below). With Python 3.8, these downgraded versions of scipy and numpy are not available anymore and I get this error, which I'm unable to fix.

1. Setup

Previous : MacOS Catalina 10.15.7, Python 2.7.16, numpy 1.9.0, scipy 1.0.1

New : MacOS Catalina 10.15.7, Python 3.8.6, numpy 1.20.3, scipy 1.4.0

2. Code

It happens when calling scipy.integrate.odeint :

y_trajectory = scipy.integrate.odeint(growth_a_derivs, y_start, t_array, atol=eps, args=myargs)

With growth_a_derivs a function, y_start, t_array numpy arrays with dtype float64, eps a float and myargs a tuple of length 1 containing a dictionary.

3. Error

The following traceback pops :

Traceback (most recent call last):
  
  <long blah blah blah>

  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pyxcosmo/icosmo_cosmo.py", line 149, in growth_a
    result=growth_a_ode(cosmo, a, unnorm=unnorm)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pyxcosmo/icosmo_cosmo.py", line 671, in growth_a_ode
    y_trajectory = scipy.integrate.odeint(growth_a_derivs, y_start, t_array, atol=eps, args=myargs)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/scipy/integrate/odepack.py", line 242, in odeint
    output = _odepack.odeint(func, y0, t, args, Dfun, col_deriv, ml, mu,
TypeError: Cannot cast array data from dtype('O') to dtype('float64') according to the rule 'safe'

So the last line is in scipy's integrate pckg, the full call to odeint is :

    output = _odepack.odeint(func, y0, t, args, Dfun, col_deriv, ml, mu,
                         full_output, rtol, atol, tcrit, h0, hmax, hmin,
                         ixpr, mxstep, mxhnil, mxordn, mxords,
                         int(bool(tfirst)))

Inside this, the error is in a compiled scipy .so file, thus I can't analyze this error further. I just know that the call to growth_a_derivs isn't problematic (its print() calls all work). But I don't know whether its a numpy or scipy problem, and how to overcome it.

4. Resolution !

I was able to understand the problem thanks to @RFoxtea answer below. I believe this issue was caused by the change in numpy version. The output of growth_a_derivs is :

np.array([f1,f2])

with :

type(f1) = numpy.float64
type(f2) = numpy.ndarray
f2.shape = (1,)

With numpy 1.9, this gives :

np.array([f1,f2]) = [1.234, np.array([1.234])]
np.array([f1,f2]).dtype = numpy.float64

With numpy 1.20, this gives :

np.array([f1,f2]) = [1.234, np.array([1.234])]
np.array([f1,f2]).dtype = numpy.object

And that is not accepted by scipy.integrate.odeint. However, notice that scipy.integrate.solve_ivp is ok with that.

Thanks for your help !

Upvotes: 0

Views: 232

Answers (2)

16Aghnar
16Aghnar

Reputation: 113

Thanks to @RFoxtea who provided the right solution.

However there is an interesting alternative. You can use scipy.integrate.solve_ivp instead of scipy.integrate.odeint.

Solve_ivp can handle an output of funthat is with dtype object.

Upvotes: 0

Raf Vost&#233;
Raf Vost&#233;

Reputation: 128

At some point in running scipy.integrate.odeint, Numpy is told to convert an array of Python objects into an array of floats, and it's answering you that it can't do that. Your description suggests the problem has to be with y_start, t_start or (maybe, I'm not sure) the return value of growth_a_derivs. Please make sure that these all have an appropriate dtype that can be converted into a float. Could be very easy to fix.

Upvotes: 1

Related Questions