Error encoutered: HTTP 400 (InvalidRequest) The deployed GPT model does not support Vision Enhancement and On Your Data (OYD) with images

Question

I am currently working on .NET to create a chat bot that accesses data from my Azure AI Search Index. My index is only composed of text and no images. I followed the steps specified in the Azure documentation (link: https://learn.microsoft.com/en-us/azure/ai-services/openai/use-your-data-quickstart?tabs=command-line%2Cpython-new&pivots=programming-language-csharp) And chose the without Streaming option, but I get the following error: HTTP 400 (InvalidRequest) The deployed GPT model does not support Vision Enhancement and On Your Data (OYD) with images.

#pragma warning disable AOAI001
    public async Task<string> SearchIndex(string IndexName, string prompt)
    {

        _searchClient = _searchIndexClient.GetSearchClient(IndexName);

        _options = new();
        
        _options.AddDataSource(new AzureSearchChatDataSource()
        {
            Endpoint = new Uri(_searchEndpoint),
            IndexName = IndexName,
            Authentication = DataSourceAuthentication.FromApiKey(_searchKey),
            InScope = true,
        });

        try
        {

            **ChatCompletion completion = _chatClient.CompleteChat([new UserChatMessage(prompt)], _options);**

            AzureChatMessageContext onYourDataContext = completion.GetAzureMessageContext();

            if (onYourDataContext?.Intent is not null)
            {

                Console.WriteLine($"Intent: {onYourDataContext.Intent}");

            }
            foreach (AzureChatCitation citation in onYourDataContext?.Citations ?? [])
            {

                Console.WriteLine($"Citation: {citation.Content}");

            }

            return completion.Content[0].Text;

        }

        catch (Exception ex)
        {

            Console.WriteLine(ex.Message.ToString());
            return ex.Message.ToString();

        }

    }

The error originates from the CompleteChat call.

I would appreciate your help on this issue!

The goal of the method is to answer user prompt using a specific index.

Answer 1

I have tried your code and got the same error with Provisioned deployment .

The error is due to the fact that gpt Version: turbo-2024-04-09 is available for both standard and provisioned deployments. Currently the provisioned version of this model doesn't support image/vision inference requests.

Provisioned deployments of this model only accept text input. Standard model deployments accept both text and image/vision inference requests. I Used Standard model deployments to reslove this error .

For more information, refer to this MSDOC and git on provisioned managed availability.

For detailed information on these services, refer to the following resources:

• Computer Vision API Documentation and Custom Vision Service Documentation.

Below is a sample code snippet for querying Azure Search and then using Azure OpenAI to process the results. The code reference is taken from MSDOC:

            string azureOpenAIEndpoint = "AZURE_OPENAI_ENDPOINT";
               
            string azureOpenAIKey ="AZURE_OPENAI_API_KEY";
            
            string deploymentName = "AZUREOPENAIDEPLOYMENTIDdetails";
            string searchEndpoint = "AZURE_AISEARCHENDPOINTName";
            string searchKey = "AZUREAISEARCHAPIKEY";
            string searchIndex = "AZUREAISEARCHINDEXName";

           
            AzureOpenAIClient azureClient = new(new Uri(azureOpenAIEndpoint), new AzureKeyCredential(azureOpenAIKey));
            ChatClient chatClient = azureClient.GetChatClient(deploymentName);

           
            string searchQuery = "health plans";
            string searchResults = await QueryAzureSearchAsync(searchEndpoint, searchKey, searchIndex, searchQuery);

           
            var chatMessage = new UserChatMessage($"Based on your search, here are the details: {searchResults}");

         
            ChatCompletionOptions options = new();

            try
            {
             
                ChatCompletion completion = await chatClient.CompleteChatAsync(new[] { chatMessage }, options);

                
                Console.WriteLine("Response:");
                Console.WriteLine(completion.Content[0].Text);

                
                AzureChatMessageContext onYourDataContext = completion.GetAzureMessageContext();


                if (onYourDataContext?.Intent != null)
                {
                    Console.WriteLine($"Intent: {onYourDataContext.Intent}");
                }

                foreach (AzureChatCitation citation in onYourDataContext?.Citations ?? Array.Empty<AzureChatCitation>())
                {
                    Console.WriteLine($"Citation: {citation.Content}");
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine($"An error occurred: {ex.Message}");
            }
        }

        private static async Task<string> QueryAzureSearchAsync(string searchEndpoint, string searchKey, string searchIndex, string query)
        {
            var client = new SearchClient(new Uri(searchEndpoint), searchIndex, new AzureKeyCredential(searchKey));
            var searchOptions = new SearchOptions { IncludeTotalCount = true };
            var response = await client.SearchAsync<SearchDocument>(query, searchOptions);

          
            var results = response.Value.GetResults().Select(result => result.Document.ToString());
            return string.Join("\n", results);
        

Output: