CODEWITHSUNDEEP
azureazure-language-understandingazure-marketplaceazure-ai
Jayendran
Jayendran

Reputation: 10960

How can we enable private endpoint for Pay as you go Model Endpoints in Azure AI

Following Meta Llama 2 7B product in Azure marketplace, I deployed it in azure ai service. AFter the deployment I got my endpoint as something like

https://mydemo-llama-serverless.eastus2.inference.ai.azure.com

I already enabled the private endpoint for my Azure AI but those endpoints can give the private endpoints for the domain privatelink.api.azureml.ms and privatelink.notebooks.azure.net however my Pay-as-you-go deployment having the domain as inference.ai.azure.com which is still public.

Question: How can I make my inference endpoint for my Pay-as-you-go model deployment as private?

If the answer is not yet supported can someone guide me on any workaround or when we can expect this feature?

Upvotes: 1

Views: 294

Answers (1)

Jayendran
Jayendran

Reputation: 10960

Finally, Microsoft has released this feature around July, read more here

Network isolation for models deployed via Serverless APIs

I tried the solution and noticed one additional limitation will be

If an AI hub is disabled public network, It cannot able to reach the marketplace to subscribe the models. Please see my below error

Error from AI Hub

To be exact the API which is failing is

https://ai.azure.com/api/<region>/modelEndpointDiscovery/v1.0/subscriptions/<>/resourceGroups/<>/providers/Microsoft.MachineLearningServices/workspaces/demoproject/marketplaceSubscriptions?api-version=2023-10-01-preview

with 403 Forbidden

To bypass this error, I have to make the Hub publicly accessible then subscribe to the model from the marketplace, and then deploy after all this is done. I have to revert my hub to disable public access after subscribing from the marketplace

Upvotes: 0

Related Questions