CODEWITHSUNDEEP
amazon-sagemakerendpointamazon-sagemaker-studio
ChunkyT
ChunkyT

Reputation: 43

How to access sagemaker model registry metrics from within the endpoint

I am trying to use amazon sagemaker to build an enpoint so I can make inferences from a trained model. The model I am using is in the model registry and has metrics associated with it. I want the endpoint to return a dataframe with two columns ['model r2', 'model_prediction']. I have currently successfully built and queried an enpoint that gives 'model_prediction' using https://github.com/aws/sagemaker-inference-toolkit. However, I don't know how to access the enpoint's model's "model quality" metrics. The models r2 is stored in the "model quality" section of the model version in the model registry and I can see the values in sagemaker studio. I feel like there is likely a 1-2 line code to return this value but I can't find anything in the sagemaker documentation. What I want would look something like this within the InferenceHandler

# See https://github.com/aws/sagemaker-inference-toolkit for more details on implementing a handler.
class InferenceHandler(DefaultInferenceHandler):
    def default_model_fn(self, model_dir):
        """
        Deserialize and return fitted model.
        """
        model = joblib.load(model_dir+"/model.joblib")
        return model
        #raise NotImplementedError

    def default_predict_fn(self, input_data, model):
        """
        SageMaker model server invokes `predict_fn` on the return value of `input_fn`.

        Args:
            input_data
            model

        Returns: predictions based on the input data using the fitted model
        """
        output = model.predict(features)
        ##### HERE
        model_metric = model.model_metrics['r2']
        ###### HERE
        return pd.DataFrame({'model_r2':model_metric, 'model_prediction':output})

I tried searching sagemaker documentation but could not find a solution

Upvotes: 1

Views: 776

Answers (1)

Kirit Thadaka
Kirit Thadaka

Reputation: 507

You can call the DescribeModelPackage API to get the model metrics - https://docs.aws.amazon.com/cli/latest/reference/sagemaker/describe-model-package.html

Upvotes: 0

Related Questions