Metric names for the GANs loss in Keras

Question

I am working with the improved version of Wasserstein GANs, for generated images when given as input some features X_{S} coupled together with noise vector z. Furthermore, I would like to generate samples from specific classes. Therefore, a classifier was added to the standard training of the whole system. My code for building the whole system is the following:

class WGANGP():
def __init__(self):

    self.target_mod = ".."
    self.learning_param = 0.0001
    self.no_input_feats = ...

    # Following parameter and optimizer set as recommended in paper
    self.n_critic = 15
    optimizer = RMSprop(lr=self.learning_param)

    self.img_rows = ...
    self.img_cols = ...
    self.channels = 3
    self.img_shape = (self.img_rows, self.img_cols, self.channels)
    self.latent_dim = ...

    # Build the generator and critic
    self.generator = build_generator(self.latent_dim, self.channels)
    #self.generator = self.build_generator_old()
    self.critic = build_critic(self.img_shape)

    #-------------------------------
    # Construct Computational Graph
    #       for the Critic
    #-------------------------------

    # Freeze generator's layers while training critic
    self.generator.trainable = False

    # Image input (real sample)
    real_img = Input(shape=self.img_shape)

    # Noise input
    z_disc = Input(shape=(self.latent_dim,))
    # Generate image based of noise (fake sample)
    fake_img = self.generator(z_disc)

    # Discriminator determines validity of the real and fake images
    fake, aux1 = self.critic(fake_img)
    valid, aux2 = self.critic(real_img)

    # Construct weighted average between real and fake images
    interpolated_img = RandomWeightedAverage()([real_img, fake_img])
    # Determine validity of weighted sample
    validity_interpolated, aux3 = self.critic(interpolated_img)

    # Use Python partial to provide loss function with additional
    # 'averaged_samples' argument
    partial_gp_loss = partial(self.gradient_penalty_loss,
                      averaged_samples=interpolated_img)

    partial_gp_loss.__name__ = 'gradient_penalty' 
    # Keras requires function names

    self.critic_model = Model(inputs=[real_img, z_disc],
                        outputs=[valid, fake, validity_interpolated, aux1])

    self.critic_model.compile(loss=[self.wasserstein_loss,
            self.wasserstein_loss,
            partial_gp_loss,
            'categorical_crossentropy'],
        optimizer=optimizer,
        metrics=['accuracy'],
        loss_weights=[1, 1, 5, 1])
    #-------------------------------
    # Construct Computational Graph
    #         for Generator
    #-------------------------------

    # For the generator we freeze the critic's layers
    self.critic.trainable = False
    self.generator.trainable = True

    # Sampled noise for input to generator
    z_gen = Input(shape=(self.latent_dim,))
    # Generate images based of noise
    img = self.generator(z_gen)
    # Discriminator determines validity
    valid = self.critic(img)
    # Defines generator model
    self.generator_model = Model(z_gen, valid)
    self.generator_model.compile(loss=self.wasserstein_loss, optimizer=optimizer)

When I am printing the critic_model.metric_names I am receiving the following:

['loss', 'model_2_loss', 'model_2_loss', 'model_2_loss', 'model_2_loss', 'model_2_acc', 'model_2_acc_1', 'model_2_acc_2', 'model_2_acc_3']

Can anyone help me understand what these names stand for?

Answer 1

The answer is right here:

self.critic_model = Model(inputs=[real_img, z_disc],
                        outputs=[valid, fake, validity_interpolated, aux1]) #<- 4 outputs

                                                              #4 model losses + 1 total loss:
self.critic_model.compile(loss=[self.wasserstein_loss,        #loss for output 0
                                self.wasserstein_loss,        #loss for output 1
                                partial_gp_loss,              #loss for output 2
                                'categorical_crossentropy']   #loss for output 3
                          optimizer=optimizer,
                          metrics=['accuracy'],           #replicated, one for each output
                          loss_weights=[1, 1, 5, 1])

Your model clearly has 4 outputs, and you defined one loss per output. Whenever you have multiple losses, Keras will sum a total loss for you, so:

• 'loss' is the total loss (sum of all losses for this model)

The other 4 'model_2_loss' are, in order:

• self.wasserstein_loss, for the first output valid
• self.wasserstein_loss, for the second output fake
• partial_gp_loss, for validity_interpolated
• 'categorical_crossentropy' for aux1

For the metrics, since you defined only one, the system replicated this same metric for each of the model's outputs:

• 'model_2_acc', metric for valid
• 'model_2_acc_1', metric for fake
• 'model_2_acc_2', metric for validity_interpolated
• 'model_2_acc_3', metric for aux1

---

For better loss names, you should add names to the outputs of the models, to the losses, etc., wherever it's possible to add a name parameter.

Some operations accept names, such as: How can I set the name of my loss operation in Tensorflow?

Losses in newer versions, created as objects, also accept names: https://www.tensorflow.org/api_docs/python/tf/keras/losses/CategoricalCrossentropy

Models accept names, such as:

self.critic_model = Model(inputs=[real_img, z_disc],
                          outputs=[valid, fake, validity_interpolated, aux1], 
                          name='critic_model')

Layers accept names, so you should have a name for the output layer of each model to keep track of things better.

I'm not totally familiar with the new eager mode execution concept, but you might try to add a name also whenever you call a model... not sure whether this is possible.

Answer 2

You have two metrics the loss and the accuracy, keras generate the name of the metrics from layer where the outputs of your model are last defined. The outputs of your model are outputs=[valid, fake, validity_interpolated, aux1] so you have four of them and all are define with the critic model:

 fake, aux1 = self.critic(fake_img)
 valid, aux2 = self.critic(real_img)
 validity_interpolated, aux3 = self.critic(interpolated_img)

so all are named name_of_the_model_name_of_the_loss -> Model_2_loss

This show how to add a name parameter to a layer to change the name given to the metric: Is there a way of renaming the metrics and losses of a Keras model?

edit on comment:

The source code from keras is here: https://github.com/tensorflow/tensorflow/blob/v2.1.0/tensorflow/python/keras/engine/training.py#L81-L2865

as shown in metrics_names function below, the first name is hard coded as 'loss' by default, probably because their is always at least a loss, and then name attribute is added automatically:

     @property
      def metrics_names(self):
        """Returns the model's display labels for all outputs."""

        # This property includes all output names including `loss` and per-output
        # losses for backward compatibility.
        metrics_names = ['loss']
        if self._is_compiled:
          # Add output loss metric names to the metric names list.
          if len(self._training_endpoints) > 1:
            metrics_names.extend([
                e.loss_name()
                for e in self._training_endpoints
                if not e.should_skip_target()
            ])

        # Add all metric names.
        metrics_names += [m.name for m in self.metrics]
        return metrics_names

The method return the self.name attribute of the losses and not a variable name that could be used with to call them

You can also use this simple code to see that metrics_names depend on last definition of models outputs:

input_ =  keras.layers.Input(shape=(8,))
x =  keras.layers.Dense(16)(input_)
output1 = keras.layers.Dense(32)(x)
output2 = keras.layers.Dense(32, name="output2")(x)
model = keras.models.Model(inputs=input_, outputs=[output1, output2])
model.compile(loss=["mse", "mae"], optimizer="adam", metrics={"output2":"accuracy"})
print(model.metrics_names)

>>> ['loss', 'dense_3_loss', 'output2_loss', 'output2_acc']