CODEWITHSUNDEEP
azureopenai-apilangchain
Neel
Neel

Reputation: 625

langchain - Azure OpenAI api - returning additional information than the asked question

I am trying to develop a chatbot using streamlit,langchain Azure OpenAI api. I have been successful in deploying the model and invoking an response but it is not what I expect. here is the prompt and the code that to invoke the API

llm = AzureOpenAI(model_name="gpt-35-turbo",
                  temperature=0,
                  openai_api_key=open_api_key,
                  openai_api_base=openai_api_base,
                  openai_api_type=openai_api_type,
                  openai_api_version=openai_api_version,
                  deployment_name=openai_deployment) 

embedding = OpenAIEmbeddings(openai_api_key=open_api_key,
                             deployment=open_ai_embed_model,
                             openai_api_base=openai_api_base,
                             openai_api_type=openai_api_type,
                             )
prompt_template = """Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.
Just answer the question asked. do not provide additional question and helpful answers
{context}

Question: {question}
"""
PROMPT = PromptTemplate(
    template=prompt_template, input_variables=["context", "question"]
)

Code to invoke the chain and pass the prompt

chain_type_kwargs = {"prompt": PROMPT}
qa_chain = RetrievalQA.from_chain_type(
       llm,
       retriever=retriever,
       chain_type="stuff",
       chain_type_kwargs=chain_type_kwargs
)
result = qa_chain.run(user_input)

now if I ask it a question such as

What is the limit of breakfast

I get the correct response such as

A: 20$

But the issue is after that I get bunch of additional information in a question answer format which although is produced using the context but not requested such as

Question: What is the limit for Laundary? Answer: 10$ Question: Question: What is the guideline for selecting flights for domestic flights? Answer: Helpful answer

It seems like model is trying to use maximum tokens allowed despite clear instruction in prompt to not provide any additional information. How can I control this any inputs will be helpful?

P.S: it work properly if I use OpenAI model rather than Azure OpenAI model

Upvotes: 1

Views: 3694

Answers (1)

Gaurav Mantri
Gaurav Mantri

Reputation: 136276

Please try by replacing “AzureOpenAI” with “AzureChatOpenAI”. I ran into this similar problem and making this change solved my problem.

Upvotes: 6

Related Questions