LangChain Chat History

Question

I am struggling with passing context to conversational rag chain when using RunnableWithMessageHistory.

I have the following query function:

def query(query_text, prompt, session_id, metadata_context):
# History retrieval test
contextualize_q_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", contextualize_q_system_prompt),
        ("system", "{context}"),
        ("system", prompt),
        MessagesPlaceholder("chat_history"),
        ("human", "{input}"),
    ]
)
history_aware_retriever = create_history_aware_retriever(
    llm, retriever, contextualize_q_prompt
)

qa_prompt = ChatPromptTemplate.from_messages(
    [   
        ("system", PROMPT_TEMPLATE),
        ("system", "{context}"),
        ("system", prompt),
        MessagesPlaceholder("chat_history"),
        ("human", "{input}"),
    ]
)

question_answer_chain = create_stuff_documents_chain(llm, qa_prompt)
rag_chain = create_retrieval_chain(history_aware_retriever, question_answer_chain)

conversational_rag_chain = RunnableWithMessageHistory(
    rag_chain,
    get_session_history,
    input_messages_key="input",
    history_messages_key="chat_history",
    output_messages_key="answer",
)
try:
    logger.info(f"Model: {LLM_MODEL} assigned. Generation of response has started.")
    response = conversational_rag_chain.invoke({"input": query_text, "context": metadata_context}, config={"configurable": {"session_id": f"{session_id}"}},)
    logger.info(f"Response generated.")
except Exception as e:
    return ({'Generation of response failed: ': str(e)})
return response["answer"]

I want to pass my own 'context' that is prepared and parsed from retriever. I do not want retriever to be called again but from what I've read - retrieving happens by itself if chat_history does not contain the answer.

prompt variable is created:

prompt_template = ChatPromptTemplate.from_template(PROMPT_TEMPLATE)
prompt = prompt_template.format(context=metadata_context, input=query_text)

As you can see I am trying to put the context everywhere but no success.

The 'context' I can see when calling

conversational_rag_chain.invoke({"input": query_text, "context": metadata_context}, config={"configurable": {"session_id": f"{session_id}"}},)
        logger.info(f"Response generated.")

is the result of retriever:

Document(metadata={'number_of_reviews': '16', 'price': 18999, 'product_name': 'product', 'rating': '4')

The code I'm using is as follows:

chroma_client = chromadb.HttpClient(host=DB_HOST, port=DB_PORT)
chroma_collection = chroma_client.get_collection(os.getenv("DB_COLLECTION"))

vectorstore = VStoreChroma(DB_COLLECTION, embedding_function, client=client)


llm = ChatOpenAI(model="gpt-4o-mini",temperature=0)

retriever = SelfQueryRetriever.from_llm(
    llm,
    vectorstore,
    document_content_description,
    metadata_field_info,
    search_kwargs = {"k": 10}
)

def self_query(query_text):
    model = llm
    logger.info("Data retrieval has started.")
    try:
        result = retriever.invoke(query_text)
        logger.info("Data retrieved from database.")
        if len(result) == 0:
            logger.info(f"Unable to find matching results.")
    except Exception as e:
        return ({'Retrieval failed: ': str(e)})
    return result

Retrieving is alright I get correct results. The problem is that the context I prepare from metadata by parsing it with the function like the one you mention in your snippet. It is string and I do not get it where I can pass it so the context is used properly. The rest is as I mentioned before.

Answer 1

You can pass your context or your question however you decide but i think fundamentally, your context should be separate from question

You can also pass your context easily using this template I have provided

    chat_template = """
                Answer the following questions{question} \n
                Based on the data and context provided {context} \n
                Question: {question} \n
            """

    # get the chat prompt template
    prompt = ChatPromptTemplate.from_template(chat_template)

To something like this below.

from langchain.prompts import ChatPromptTemplate
from langchain.chat_models import ChatOpenAI
from langchain.chains import LLMChain
from langchain_core.output_parsers import StrOutputParser
from operator import itemgetter

prompt = ChatPromptTemplate.from_template(chat_template)

# Initialize the LLM
llm = ChatOpenAI(temperature=0.5, max_tokens=4096)

# Initialize the output parser
output_parser = StrOutputParser()

# Create the LLMChain
final_rag = LLMChain(prompt=prompt, llm=llm, output_parser=output_parser)

# Invoke the chain with the input
question = "Your question here"  # Replace this with your actual question
response = final_rag.invoke({"input": question, "context": itemgetter(metadata_context)})

print(response)