Reputation: 424
I am trying to chat with Azure OpenAI models using my own data. I have uploaded my PDF documents into Azure Blob Storage and created a search index for them. I have a C# application where I’m trying to stream an LLM’s (GPT-4) response to the frontend. I am referring to an article, and my question is very similar to one discussed there.
I want to allow the user to open the relevant documents that were used by the model to generate the response. I understand that to do this, I need to get the “Citations” from the ChatCompletion’s response. I was able to view and access the citations when I didn’t use streaming. However, with streaming on, I can’t seem to find the citations key. Here’s my code:
AzureOpenAIClient azureClient = new(
new Uri(_endpoint),
new AzureKeyCredential(_apiKey));
ChatClient chatClient = azureClient.GetChatClient(_deploymentName);
ChatCompletionOptions options = new()
{
Temperature = (float)temperature,
MaxTokens = maxTokens,
FrequencyPenalty = (float)frequencyPenalty,
PresencePenalty = (float)presencePenalty
};
options.AddDataSource(new AzureSearchChatDataSource()
{
Endpoint = new Uri(_searchEndpoint),
IndexName = _searchIndex,
Authentication = DataSourceAuthentication.FromApiKey(_searchKey),
VectorizationSource = Azure.AI.OpenAI.Chat.DataSourceVectorizer.FromDeploymentName("text-embedding-ada-002")
});
var chatUpdates = chatClient.CompleteChatStreamingAsync(
[
new UserChatMessage(fullInput),
], options);
AzureChatMessageContext onYourDataContext = null;
var streamedContentParts = new List<string>();
Console.Write($"CHATUPDATES:::::::::::: {chatUpdates}");
await foreach (var chatUpdate in chatUpdates)
{
string chatUpdateJson = JsonSerializer.Serialize(chatUpdate, new JsonSerializerOptions { WriteIndented = true });
Console.Write($"CHATUPDATEJSON:::::::::::: {chatUpdateJson}");
if (chatUpdate.Role.HasValue)
{
continue;
}
foreach (var contentPart in chatUpdate.ContentUpdate)
{
var jsonResponse = JsonSerializer.Serialize(new { Text = contentPart.Text });
streamedContentParts.Add(contentPart.Text);
yield return jsonResponse;
}
if (onYourDataContext == null)
{
onYourDataContext = chatUpdate.GetAzureMessageContext();
}
onYourDataContext = chatUpdate.GetAzureMessageContext();
string contextJson = JsonSerializer.Serialize(onYourDataContext, new JsonSerializerOptions { WriteIndented = true });
Console.WriteLine($"CONTEXTJSON:::::::::{contextJson}");
}
foreach (AzureChatCitation citation in onYourDataContext?.Citations ?? [])
{
Console.Write($"Citation: {citation.Content}");
}
I added some comments to debug the output. Here's a snapshot of my output:
As you can see "onYourDataContext" is null and the looping through AzureChatCitation citation is also not giving out any outputs.
Upvotes: 1
Views: 432