CODEWITHSUNDEEP
numpyrandomscikit-learndistributiongrid-search
O.rka
O.rka

Reputation: 30677

sklearn's `RandomizedSearchCV` not working with `np.random.RandomState`

I am trying to optimize a pipeline and wanted to try giving RandomizedSearchCV a np.random.RandomState object. I can't it to work but I can give it other distributions.

Is there a special syntax I can use to give RandomSearchCV a np.random.RandomState(0).uniform(0.1,1.0)? 

from scipy import stats
import numpy as np
from sklearn.neighbors import KernelDensity
from sklearn.grid_search import RandomizedSearchCV

# Generate data
x = np.random.normal(5,1,size=int(1e3))

# Make model
model = KernelDensity()

# Gridsearch for best params
# This one works
search_params = RandomizedSearchCV(model, param_distributions={"bandwidth":stats.uniform(0.1, 1)}, n_iter=30, n_jobs=2)
search_params.fit(x[:, None])

# RandomizedSearchCV(cv=None, error_score='raise',
#           estimator=KernelDensity(algorithm='auto', atol=0, bandwidth=1.0, breadth_first=True,
#        kernel='gaussian', leaf_size=40, metric='euclidean',
#        metric_params=None, rtol=0),
#           fit_params={}, iid=True, n_iter=30, n_jobs=2,
#           param_distributions={'bandwidth': <scipy.stats._distn_infrastructure.rv_frozen object at 0x106ab7da0>},
#           pre_dispatch='2*n_jobs', random_state=None, refit=True,
#           scoring=None, verbose=0)

# This one doesn't work :(
search_params = RandomizedSearchCV(model, param_distributions={"bandwidth":np.random.RandomState(0).uniform(0.1, 1)}, n_iter=30, n_jobs=2)
# TypeError: object of type 'float' has no len()

Upvotes: 1

Views: 2014

Answers (1)

sascha
sascha

Reputation: 33512

What you observe is expected, as the class-method uniform of an object of type np.random.RandomState() immediately draws a sample at the time of the call.

Compared to that, your usage of scipy's stats.uniform() creates a distribution yet to sample from. (Although i'm not sure if it's working as you expect in your case; be careful with the parameters).

If you want to incorporate something based on np.random.RandomState() you have to build your own class like mentioned in the docs:

This example uses the scipy.stats module, which contains many useful distributions for sampling parameters, such as expon, gamma, uniform or randint. In principle, any function can be passed that provides a rvs (random variate sample) method to sample a value. A call to the rvs function should provide independent random samples from possible parameter values on consecutive calls.

Upvotes: 2

Related Questions