megh_sat
megh_sat

Reputation: 424

"AzureChatCitation citation" is not working with the CompleteChatStreamingAsync streamer on

I am trying to chat with Azure OpenAI models using my own data. I have uploaded my PDF documents into Azure Blob Storage and created a search index for them. I have a C# application where I’m trying to stream an LLM’s (GPT-4) response to the frontend. I am referring to an article, and my question is very similar to one discussed there.

I want to allow the user to open the relevant documents that were used by the model to generate the response. I understand that to do this, I need to get the “Citations” from the ChatCompletion’s response. I was able to view and access the citations when I didn’t use streaming. However, with streaming on, I can’t seem to find the citations key. Here’s my code:

 AzureOpenAIClient azureClient = new(
     new Uri(_endpoint),
     new AzureKeyCredential(_apiKey));
 ChatClient chatClient = azureClient.GetChatClient(_deploymentName);

 ChatCompletionOptions options = new()
 {
     Temperature = (float)temperature,
     MaxTokens = maxTokens,
     FrequencyPenalty = (float)frequencyPenalty,
     PresencePenalty = (float)presencePenalty
 };
 options.AddDataSource(new AzureSearchChatDataSource()
 {
     Endpoint = new Uri(_searchEndpoint),
     IndexName = _searchIndex,
     Authentication = DataSourceAuthentication.FromApiKey(_searchKey),
     VectorizationSource = Azure.AI.OpenAI.Chat.DataSourceVectorizer.FromDeploymentName("text-embedding-ada-002")
   });
 var chatUpdates = chatClient.CompleteChatStreamingAsync(
 [
 new UserChatMessage(fullInput),
     ], options);
 AzureChatMessageContext onYourDataContext = null;
 var streamedContentParts = new List<string>();
 Console.Write($"CHATUPDATES:::::::::::: {chatUpdates}");
 await foreach (var chatUpdate in chatUpdates)
 {
     string chatUpdateJson = JsonSerializer.Serialize(chatUpdate, new JsonSerializerOptions { WriteIndented = true });
     Console.Write($"CHATUPDATEJSON:::::::::::: {chatUpdateJson}");
     if (chatUpdate.Role.HasValue)
     {
         continue;
     }
     foreach (var contentPart in chatUpdate.ContentUpdate)
     {
         var jsonResponse = JsonSerializer.Serialize(new { Text = contentPart.Text });
         streamedContentParts.Add(contentPart.Text);
         yield return jsonResponse;
     }
     if (onYourDataContext == null)
     {
         onYourDataContext = chatUpdate.GetAzureMessageContext();
     }
     onYourDataContext = chatUpdate.GetAzureMessageContext();
     string contextJson = JsonSerializer.Serialize(onYourDataContext, new JsonSerializerOptions { WriteIndented = true });
     Console.WriteLine($"CONTEXTJSON:::::::::{contextJson}");
 }
 foreach (AzureChatCitation citation in onYourDataContext?.Citations ?? [])
 {
     Console.Write($"Citation: {citation.Content}");
 }

I added some comments to debug the output. Here's a snapshot of my output:

enter image description here As you can see "onYourDataContext" is null and the looping through AzureChatCitation citation is also not giving out any outputs.

Upvotes: 1

Views: 432

Answers (0)

Related Questions