CODEWITHSUNDEEP
evaluationgoogle-cloud-vertex-ai
星空の物語
星空の物語

Reputation: 1

How can I change the judge model of Gen AI evaluation service?

From this link, we can see

The Gen AI evaluation service uses gemini-1.5-pro as a judge model

But I want to customize judge model by myself, for example, using Claude. I didn't find some instruction about it.

And then I found an official document in Github: Customize Model-based Metrics to Evaluate a Gen AI model . And there's a instruction named Create a fully-customized PointwiseMetric. I'm not sure if I can use model = GenerativeModel("model_version", generation_config=generation_config) to generate a model and then assign it as my judge model by free_form_pointwise_result = free_form_pointwise_eval_task.evaluate(model=model).

Dose anybody know about this?

Upvotes: 0

Views: 29

Answers (0)

Related Questions