How to initialize the variables of the existing Keras Models?

Question

I would like to retrain Keras Model, Inception_v3, from scratch.

The model is defined here: https://github.com/keras-team/keras-applications/blob/master/keras_applications/inception_v3.py

I read some posts,

The listed solutions are:

1. Freeze the layers (This is not what I want...)

   for layer in model.layers:
       layer.trainable = False
   

2. https://stackoverflow.com/a/51727616/7748163

Reset all layers by checking for initializers:

def reset_weights(model):
    session = K.get_session()
    for layer in model.layers: 
        if hasattr(layer, 'kernel_initializer'):
            layer.kernel_initializer.run(session=session)
        if hasattr(layer, 'bias_initializer'):
            layer.bias_initializer.run(session=session)     

3. Use tf.variables_initializer

     model = InceptionV3()
     for layer in model.layers:
         sess.run(tf.variables_initializer(layer.weights))
   

   Reference: https://stackoverflow.com/a/56634827/7748163

4. The best one I think, but it raises an error.

   sess = tf.Session()
   for layer in model.layers: 
        for v in layer.__dict__:
            v_arg = getattr(layer,v)
            if hasattr(v_arg,'initializer'):
                initializer_method = getattr(v_arg, 'initializer')
                initializer_method.run(session=sess)
                print('reinitializing layer {}.{}'.format(layer.name, v))
   

However, none of them works for Inception_v3.

The error information is for BatchNorm layer:

tensorflow.python.framework.errors_impl.FailedPreconditionError: Error while reading resource variable batch_normalization_9/moving_mean from Container: localhost. This could mean that the variable was uninitialized. Not found: Resource localhost/batch_normalization_9/moving_mean/N10tensorflow3VarE does not exist.
     [[{{node batch_normalization_9_1/AssignMovingAvg/ReadVariableOp}}]]
     [[metrics_1/categorical_accuracy/Identity/_469]]

So, how to re-train the existing Keras Models, and initialize the variables? What is the best practice to re-train a model from Keras applications?

Further discussion:

https://github.com/keras-team/keras/issues/341

Answer 1

Why not simply not asking for the weights?

model = Inception_V3(..., weights=None,...)

https://github.com/keras-team/keras-applications/blob/master/keras_applications/inception_v3.py/#L100