CODEWITHSUNDEEP
spring-bootopenai-apiretrieval-augmented-generationspring-ai
Sagar
Sagar

Reputation: 21

AI model answering questions outside of the context provided in the System Prompt (RAG process)

I am building an app using the RAG process. I have loaded some private documents and I am able to have a conversation with the AI model (I am using Open AI) about the data in the private documents and I am receiving satisfactory responses. The right data is being retrieved from the Vector database based on my query prompts (User Message). But, when I ask a question which is completely outside the context of the documents loaded in the Vector database, then I am still receiving a response; in-spite of no documents being retrieved from the Vector database.

For e.g., when I ask a question like 'How many planets are there in our Solar System', I still get an answer from the AI model even if 0 documents (from the vector database) is passed into it as a System prompt. Any reason why this is happening? Is there a way I can prevent the AI model from answering questions outside of the documents context?

This is my code for the Vector similarity search

private Message generateSystemMessage(String message) {
        LOGGER.info("Retrieving documents");
        List<Document> similarDocuments = vectorStore.similaritySearch(SearchRequest.query(message)
                .withTopK(2).withSimilarityThreshold(0.75));
        LOGGER.info("Found {} similar documents", similarDocuments.size());
        SystemPromptTemplate systemPromptTemplate = new SystemPromptTemplate(this.systemPromptResource);
        if(similarDocuments.isEmpty()) {
            return systemPromptTemplate.createMessage(Map.of("documents", "No information found"));
        }
        String documentContent = similarDocuments.stream().map(Document::getContent).collect(Collectors.joining("\n"));
        return systemPromptTemplate.createMessage(Map.of("documents", documentContent));
    }

This is my System Prompt

`You are a helpful assistant, conversing with a user about the subjects contained in a set of documents. Use the information from the DOCUMENTS section to provide accurate answers. If unsure or if the answer isn't found in the DOCUMENTS section, simply state that you don't know the answer. And do not answer to any question that is not related to the context provided in the DOCUMENTS section.

DOCUMENTS: {documents}`

These are the logs when I ask a question outside of the context of the documents provided

`2024-03-13T21:26:17.172+11:00 INFO 25298 --- [io-8080-exec-10] c.example.rag.qa.StreamingChatService : Retrieving documents 2024-03-13T21:26:17.714+11:00 INFO 25298 --- [io-8080-exec-10] c.example.rag.qa.StreamingChatService : Found 0 similar documents 2024-03-13T21:26:17.716+11:00 INFO 25298 --- [io-8080-exec-10] c.example.rag.qa.StreamingChatService : The system prompt is -- You are a helpful assistant, conversing with a user about the subjects contained in a set of documents. Use the information from the DOCUMENTS section to provide accurate answers. If unsure or if the answer isn't found in the DOCUMENTS section, simply state that you don't know the answer. And do not answer to any question that is not related to the context provided in the DOCUMENTS section.

DOCUMENTS: No information found `

Upvotes: 2

Views: 2069

Answers (2)

Jens Schauder
Jens Schauder

Reputation: 82008

@rdiachenko gave a good answer what you can do about this.

As to why this happens:

A generative LLM basically finds likely continuations of the text you provide. If your text contains a question it will produce an answer. For this answer it works with two sources: All the documents it was trained with and whatever you provide in the prompt.

With RAG you add stuff to the prompt that should be highly relevant to the main question. The model will "notice" that and use that content. But note, that it will still use information it got from training. The way how it constructs sentences, certain words and phrases, the way it combines facts to deduce new facts. All this comes from the training phase which bascically means a large portion of the internet.

But without any contents from your private documents it will still try to provide an answer.

Similar to an intern to which give and empty folder and ask them: "Based on these documents, what are the planets of the solar system". You will still get an answer even though there was no useful document provided.

Of course you can instruct the LLM to decline an answer when no documents are provided.

Upvotes: 0

rdiachenko
rdiachenko

Reputation: 763

You could refine your system prompt to be more restrictive, here's a good example:

You're assisting with questions about services offered by Carina.
Carina is a two-sided healthcare marketplace focusing on home care aides (caregivers)
and their Medicaid in-home care clients (adults and children with developmental disabilities and low income elderly population).
Carina's mission is to build online tools to bring good jobs to care workers, so care workers can provide the
best possible care for those who need it.
        
Use the information from the DOCUMENTS section to provide accurate answers but act as if you knew this information innately.
If unsure, simply state that you don't know.
        
DOCUMENTS:
{documents}

Alternatively, if the vector store lacks relevant documents and you want to avoid answering irrelevant questions, why call the OpenAI API in the first place? Consider revising your logic so that if no documents are found, the API isn't called, and a default message is returned instead.

Upvotes: 4

Related Questions