CODEWITHSUNDEEP
pythonscikit-learnclassificationmlp
watermelon
watermelon

Reputation: 1

what weight initialization does the mlp classifier in sklearn use?

I need to know which method of weight initialization the MLPClassifier in Sklearn uses. I know there are several ways to initialize weights in a Neural Network, for example random normal or randorm uniform or glorot. However, I did not find any information on which of these methods is used in Sklean's MLPClassifier.

Upvotes: 0

Views: 579

Answers (1)

Jack P.
Jack P.

Reputation: 136

Based on the source code, parameters are initialized with the method from this paper by Glorot et al.:

def _init_coef(self, fan_in, fan_out, dtype):
    # Use the initialization method recommended by
    # Glorot et al.
    factor = 6.0
    if self.activation == "logistic":
        factor = 2.0
    init_bound = np.sqrt(factor / (fan_in + fan_out))

    # Generate weights and bias:
    coef_init = self._random_state.uniform(
        -init_bound, init_bound, (fan_in, fan_out)
    )
    intercept_init = self._random_state.uniform(-init_bound, init_bound, fan_out)
    coef_init = coef_init.astype(dtype, copy=False)
    intercept_init = intercept_init.astype(dtype, copy=False)
    return coef_init, intercept_init

Upvotes: 1

Related Questions