Is it possible to speed up local AzureML model deployment?

Question

We want to be able to quickly test changes to entry_script.py. We can test minor changes with unit tests but we want to run a model in the context of our other backend pieces, locally.

So we run az ml model deploy with a deployment config with a computeType of LOCAL. Non-local deployment is slow, but we were hoping that local deployment would be faster. Unfortunately it isn't. In some cases it can take up to 20 minutes to deploy a model to a local endpoint.

Is there a way to speed this up for faster edit-debug loops or a better way of handling this scenario?

Few things I was thinking of:

• I was thinking az ml service update could be an option but even that takes a long time.
• Editing the file directly in the container is an option, but this is still annoying to manually synchronize with changes in your local filesystem.
• I was thinking of a folder mount in the container, but it seems there is some magic AzureML does, for example copying the entry_script.py to /var/azureml-app/main.py. We could maybe emulate this by creating a dist folder locally that matches the layout and mounting that to the container, but I'm not sure if this folder layout would change or there's other things that AzureML does.

Answer 1

Please follow the below notebook, If you want to test deploying a model rapidly you should check out https://github.com/Azure/MachineLearningNotebooks/tree/master/how-to-use-azureml/deployment/deploy-to-local

the SDK enables building and running the docker locally and updating in place as you iterate on your script to save time.