Global Fitting with Symfit: typestructure of dataset

Question

I want to perform a global fitting of datasets with shared variables using symfit. I have a numpy array xdata which is common for all datasets ydata_i which are also numpy arrays.

Following the example in the documentation a can setup the variables, the parameters and the model but I fail to setup the fit: fit = Fit(model, x_1=xdata_1, x_2=xdata_2, ..., y_1=ydata_1, y_2=ydata_2, ...)

For a small number of datasets I can write the code by hand or copy/paste it but I have hundred datasets and I hope I can avoid type in the code by hand. I tried to use lists [xdata, ydata_1, ydata2, ...] or [xdata, ydata_1, xdata, ydata2, ...] or arrays but this seems not to be the right way.

Does anyone know how the structure/type of the ordered_data should look like. Thanks

Answer 1

Sorry I need once more your help. I end up with an error wrapped_func() keywords must be strings or 'Variable' object has no attribute 'symbol'. I think it is a simple problem but I don't get the point. Would you have a look on the following example?

import numpy as np
import symfit as sf

# creating the data
freq = 10 * np.linspace(0.1,0.3,2)
phase = np.pi * np.linspace(0,0.3,2)
offset = 1.0
amplitude = 0.1

# x - array
x_array = np.arange(0,20,0.02)
# create dataset
dataset = [offset + amplitude * np.cos(freq * x_array + phase) + np.random.normal(size=len(x_array), scale=0.01) for freq,phase in zip(freq,phase)]

# independent variables
xs = sf.variables(', '.join('x_{}'.format(i) for i in range(len(dataset))))
# dependent variables
ys = sf.variables(', '.join('y_{}'.format(i) for i in range(len(dataset))))
# coupled parameters
amp, off = sf.parameters('amp, off', value=[1.0,0.1])
# decoupled parameters
freqc = sf.parameters(', '.join('f_{}'.format(i) for i in range(len(dataset))),value=freq)
phasec = sf.parameters(', '.join('p_{}'.format(i) for i in range(len(dataset))),value=phase)

# setup model
model_dict = {y : off + amp * sf.cos(freq * x + phase) for x, y, freq, phase in zip(xs, ys, freqc, phasec) }
# create dataset_dict
xdata = [x_array for i in range(len(dataset))] # just to have equal length of xdata list and y-data list
#data_dict = {x : data for x, data in zip(xs + ys, xdata + dataset)} # error 'wrapped_func() keywords must be strings'
#data_dict = {str(x) : data for x, data in zip(xs + ys, xdata + dataset)} # error 'Variable' object has no attribute 'symbol'
data_dict = {'x_0': x_array, 'x_1': x_array, 'y_0': dataset[0], 'y_1': dataset[1]} # error 'Variable' object has no attribute 'symbol'
# # do the fit
fit = sf.Fit(model_dict, **data_dict)
fit_result = fit.execute()

I use Python 3.7.4 on a Windows PC via Anconda. Symfit version: 0.4.6, Sympy version: 1.4

Traceback (most recent call last):

  File "C:/Users/dummy/Documents/Scripts/Python/Scripts/SymFit_example.py", line 42, in <module>
    fit_result = fit.execute()

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\symfit\core\fit.py", line 1537, in execute
    minimizer_ans = self.minimizer.execute(**minimize_options)

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\symfit\core\minimizers.py", line 359, in execute
    return super(ScipyGradientMinimize, self).execute(jacobian=self.wrapped_jacobian, **minimize_options)

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\symfit\core\support.py", line 355, in wrapped_func
    return func(*bound_args.args, **bound_args.kwargs)

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\symfit\core\minimizers.py", line 296, in execute
    **minimize_options

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\scipy\optimize\_minimize.py", line 594, in minimize
    return _minimize_bfgs(fun, x0, args, jac, callback, **options)

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\scipy\optimize\optimize.py", line 996, in _minimize_bfgs
    gfk = myfprime(x0)

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\scipy\optimize\optimize.py", line 326, in function_wrapper
    return function(*(wrapper_args + args))

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\scipy\optimize\optimize.py", line 756, in approx_fprime
    return _approx_fprime_helper(xk, f, epsilon, args=args)

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\scipy\optimize\optimize.py", line 690, in _approx_fprime_helper
    f0 = f(*((xk,) + args))

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\scipy\optimize\optimize.py", line 326, in function_wrapper
    return function(*(wrapper_args + args))

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\symfit\core\minimizers.py", line 273, in wrapped_func
    return np.array(func(**parameters))

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\symfit\core\support.py", line 355, in wrapped_func
    return func(*bound_args.args, **bound_args.kwargs)

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\symfit\core\objectives.py", line 151, in __call__
    evaluated_func = self.model(**jac_kwargs)

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\symfit\core\fit.py", line 334, in __call__
    return Ans(*self.eval_components(**bound_arguments.arguments))

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\symfit\core\fit.py", line 296, in eval_components
    return [expr(*args, **kwargs) for expr in self.numerical_components]

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\symfit\core\support.py", line 217, in __get__
    setattr(obj, self.cache_attr, self.fget(obj))

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\symfit\core\fit.py", line 457, in numerical_components
    return [sympy_to_py(expr, self.independent_vars, self.params) for expr in self.values()]

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\symfit\core\fit.py", line 457, in <listcomp>
    return [sympy_to_py(expr, self.independent_vars, self.params) for expr in self.values()]

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\symfit\core\support.py", line 91, in sympy_to_py
    return lambdify((vars + params), func, modules='numpy', dummify=False)

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\sympy\utilities\lambdify.py", line 767, in lambdify
    funcstr = funcprinter.doprint(funcname, args, expr)

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\sympy\utilities\lambdify.py", line 977, in doprint
    argstrs, expr = self._preprocess(args, expr)

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\sympy\utilities\lambdify.py", line 1039, in _preprocess
    s = self._argrepr(arg)

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\sympy\printing\codeprinter.py", line 100, in doprint
    lines = self._print(expr).splitlines()

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\sympy\printing\printer.py", line 287, in _print
    return getattr(self, printmethod)(expr, **kwargs)

  File "C:\Users\dummy\Anaconda3\envs\spyder-beta\lib\site-packages\sympy\printing\codeprinter.py", line 344, in _print_Variable
    return self._print(expr.symbol)

AttributeError: 'Variable' object has no attribute 'symbol'

Answer 2

For a large number of datasets you can use a dictionary:

data = {'x_1': xdata_1, 'x_2': xdata_2, ..., 'y_1': ydata_1, 'y_2': ydata_2, ...}
fit = Fit(model, **data)

This way it will end up in named_data instead, which is preferred. Good luck!

p.s. you might also want to consider using a JacobianModel or a CallableModel instead of the default model if you are working with such large models, because calculating the jacobian and hessian for such a model might be expensive and unnecessary.