CODEWITHSUNDEEP
amazon-sagemakeramazon-sagemaker-studio
arve
arve

Reputation: 801

how to run a batch transform job in sagemaker pipeline via custom inference code?

based on the aws documentation/example provided here , https://sagemaker-examples.readthedocs.io/en/latest/sagemaker-pipelines/tabular/abalone_build_train_deploy/sagemaker-pipelines-preprocess-train-evaluate-batch-transform.html#Define-a-Transform-Step-to-Perform-Batch-Transformation, a model is created and a batch transform inference can be run on the trained model. it works for this example but if we need a custom inference script, How do we include a custom inference script in the model or model package before we run the batch transformation ?

from sagemaker.transformer import Transformer
from sagemaker.inputs import TransformInput
from sagemaker.workflow.steps import TransformStep

transformer = Transformer(
    model_name=step_create_model.properties.ModelName,
    instance_type="ml.m5.xlarge",
    instance_count=1,
    output_path=f"s3://{default_bucket}/AbaloneTransform",
)

step_transform = TransformStep(
    name="AbaloneTransform", transformer=transformer, inputs=TransformInput(data=batch_data)
)

Upvotes: 3

Views: 2541

Answers (1)

Giuseppe La Gualano
Giuseppe La Gualano

Reputation: 1720

You need a "model repacking step".

From the Amazon SageMaker Workflows FAQ:

Model repacking happens when the pipeline needs to include a custom script in the compressed model file (model.tar.gz) to be uploaded to Amazon S3 and used to deploy a model to a SageMaker endpoint. When SageMaker pipeline trains a model and registers it to the model registry, it introduces a repack step if the trained model output from the training job needs to include a custom inference script. The repack step uncompresses the model, adds a new script, and recompresses the model. Running the pipeline adds the repack step as a training job.

Basically, you can redefine a sagemaker Model by calling the training output as model_data and pass the inference script as entry_point.

Then sequentially, after training the model, redefine the model by changing the entry_point and on the latter you can use the transformer.

This is an example flow taken from a tested code:

my_model = Model(
    image_uri=your_img_uri,
    model_data=step_train.properties.ModelArtifacts.S3ModelArtifacts,
    role=role,
    entry_point='inference_script.py',
    name="your_inference_step_name"
)

step_create_model = ModelStep(
    name="YourInfName",
    step_args=my_model.create(instance_type="ml.m5.xlarge")
)

transformer = Transformer(
    model_name=step_create_model.properties.ModelName,
    instance_count=your_instance_count,
    instance_type=your_instance_type,
    output_path=your_path
)

Of course, instead of the generic Model, you can directly use the one that best suits your requirements (e.g. PyTorchModel etc...).

Upvotes: 4

Related Questions