CODEWITHSUNDEEP
tensorflow-federated
Eliza
Eliza

Reputation: 584

TFF: How Can I change the evaluation function of Federated learning

To implement my code with TFF, I use the method

tff.learning.build_federated_evaluation()

But I'm not understanding how this method evaluate accuracy across clients. So, like my question indicates, I would like to change the metrics and code of this evaluation funtion in TFF, so how I can proceed, link please of code fucntion. Thanks!!

Upvotes: 1

Views: 227

Answers (1)

Zachary Garrett
Zachary Garrett

Reputation: 2941

Depending on what "change the metrics code of this evaluation function" means, possibly nothing needs to change in tff.learning.build_federated_evaluation().

If you want to add a new metric, this can be implemented on the tff.learning.Model.

  • If using a tf.keras.Model simply add the metric to the list of metrics passed to tff.learning.from_keras_model(). This will produce an example weighted metric value.
  • If not using Keras, you will need to extend the implementation of tff.learning.Model.report_local_ouputs() and tff.learning_model.federated_output_computation of your subclass. An example can be found here. Notice how the tff.learning.Model.forward_pass updates the metric variables, then tff.learnining.Model.report_local_outputs computs the final metric value across local batches. Finally tff.learning.Model.federated_output_computation provides the mechanism for computing global metric values across clients.

For reference, the implementation of tff.learning.build_federated_evaluation() can be found here and shows how the tff.learning.Model methods discussed above are used together to compute metrics.

Upvotes: 1

Related Questions