How to call an Azure OpenAI service which is behind an API Management instance?

Question

I followed this video walkthrough to set up an Azure OpenAI api management service. Then, using Azure.AI.OpenAI 2.0.0-beta.2, I'm trying to call the chat completion endpoint of a gpt-4o deployment which is behind an API Management instance - but can't.

Initially, I followed the example appearing in the AzureOpenAI-with-APIM documentation (note that the Nuget version over there is 1.0.0-beta.5, earlier then mine), which says the url should be in the following format:

var url = $"{apim_url}/deployments/gpt-4o/chat/completions?api-version=2024-02-01";

But this results in 404 Not Found error. Looking at my APIM logs, I see that the actual url used was wrong - the chat/completion part is duplicated by the client.

So I changed it to the format in the following code, which managed to stop the 404 error, but I now get a 401 error:

// apim_url is taken from my apim overview page 
var apim_url = "redacted";
    
// subscription_key is taken from the subscriptions key page
var subscription_key = "redacted";
    
var url = $"{apim_url}/deployments/gpt-4o?api-version=2024-02-01"; 
    
var openAIClient = new OpenAIClient(
    credential: new System.ClientModel.ApiKeyCredential(subscription_key),
    options: new OpenAIClientOptions() { Endpoint = new Uri(url) }
);

var chatClient = openAIClient.GetChatClient("gpt-4o");

// this results in 401 error
var completion = chatClient.CompleteChat(new ChatMessage[] {
    new SystemChatMessage("You are a helpful assistant that talks like a pirate."),
    new UserChatMessage("Hi, can you help me?"),
});

So it seems like the subscription_key is not being used. I know it should be passed as an api-key header, but how do I do that?

Answer 1

[self-answer]

There are two ways to add the subscription key as an api-key header:

Solution no. 1 - manually add api-key header

// apim_url is taken from my apim overview page 
var apim_url = "redacted";
    
// note this is not the "official" url mentioned in the documentation
var url = $"{apim_url}/deployments/gpt-4o?api-version=2024-02-01";
    
// subscription_key is taken from the subscriptions key page
var subscription_key = "redacted";
    
// create an httpClient and set subscription_key in an 'api-key' header
var httpClient = new HttpClient();
httpClient.DefaultRequestHeaders.Add("api-key", subscription_key);
var clientOptions = new OpenAIClientOptions
{
    Endpoint = new Uri(url),
    Transport = 
        // For Azure.AI.OpenAI 2.0.0, use:
        new System.ClientModel.Primitives.HttpClientPipelineTransport(httpClient)
        // For Azure.AI.OpenAI 1.0.0, use this instead:
        // new Azure.Core.Pipeline.HttpClientTransport(httpClient)
};
    
var openAIClient = new OpenAIClient(
    credential: new System.ClientModel.ApiKeyCredential(subscription_key), // still required!
    options: clientOptions
);
var chatClient = openAIClient.GetChatClient("gpt-4o");
var completion = chatClient.CompleteChat(new ChatMessage[] {
    new SystemChatMessage("You are a helpful assistant that talks like a pirate."),
    new UserChatMessage("Hi, can you help me?"),
});

Solution no. 2 - implement a "policy" class, and set the api-key header in it

• For Azure.AI.OpenAI 2.0.0:

Implement a PipelinePolicy:

public class ApiKeyPolicy : PipelinePolicy
{
    private readonly string _apiKey;

    public ApiKeyPolicy(string apiKey)
    {
        _apiKey = apiKey;
    }

    public override void Process(PipelineMessage message, IReadOnlyList<PipelinePolicy> pipeline, int currentIndex)
    {
        message.Request.Headers.Add("api-key", _apiKey);
        ProcessNext(message, pipeline, currentIndex);
    }

    public override ValueTask ProcessAsync(PipelineMessage message, IReadOnlyList<PipelinePolicy> pipeline, int currentIndex)
    {
        message.Request.Headers.Add("api-key", _apiKey);
        return ProcessNextAsync(message, pipeline, currentIndex);
    }
}

Then:

// apim_url is taken from my apim overview page 
var apim_url = "redacted";

// note this is not the "official" url mentioned in the documentation
var url = $"{apim_url}/deployments/gpt-4o?api-version=2024-02-01";

// subscription_key is taken from the subscriptions key page
var subscription_key = "redacted";

// create an apiKeyPolicy and set clientOptions with it
var apiKeyPolicy = new ApiKeyPolicy(subscription_key);
var clientOptions = new OpenAIClientOptions 
{ 
    Endpoint = new Uri(url), 
    Transport = new System.ClientModel.Primitives.HttpClientPipelineTransport(new HttpClient()) 
};
clientOptions.AddPolicy(apiKeyPolicy, PipelinePosition.PerCall);

var openAIClient = new OpenAIClient(
    credential: new System.ClientModel.ApiKeyCredential(subscription_key), // still required!
    options: clientOptions
);
var chatClient = openAIClient.GetChatClient("gpt-4o");
var completion = chatClient.CompleteChat(new ChatMessage[] {
    new SystemChatMessage("You are a helpful assistant that talks like a pirate."),
    new UserChatMessage("Hi, can you help me?"),
});

• For Azure.AI.OpenAI 1.0.0:

Implement an HttpPipelinePolicy:

public class ApiKeyPolicy : HttpPipelinePolicy
{
    private readonly string _apiKey;

    public ApiKeyPolicy(string apiKey)
    {
        _apiKey = apiKey;
    }

    public override void Process(HttpMessage message, ReadOnlyMemory<HttpPipelinePolicy> pipeline)
    {
        message.Request.Headers.Add("api-key", _apiKey);
        ProcessNext(message, pipeline);
     }

     public override ValueTask ProcessAsync(HttpMessage message, ReadOnlyMemory<HttpPipelinePolicy> pipeline)
     {
         message.Request.Headers.Add("api-key", _apiKey);
         return ProcessNextAsync(message, pipeline);
     }
}

Then:

// ... same code as above ...
var clientOptions = new OpenAIClientOptions 
{ 
    Endpoint = new Uri(url), 
    Transport = new Azure.Core.Pipeline.HttpClientTransport(new HttpClient()) 
};
clientOptions.AddPolicy(apiKeyPolicy, HttpPipelinePosition.PerCall);
// ... same code as above ...