CODEWITHSUNDEEP
openai-apichatgpt-api
TaeWoo
TaeWoo

Reputation: 481

ChatGPT - a way to determine if a response is "not unknown"?

Right now, there doesn't seem to be a way of determining if response from chatgpt is vague / not known without checking the response content.

    import openai
    
    # Set up OpenAI API client
    openai.api_key = 'YOUR_API_KEY'
    
    # Define your prompt and additional messages
    prompt = '...'
    messages = [{'role': 'system', 'content': 'You are a helpful assistant.'},
                {'role': 'user', 'content': 'What is the capital of France?'}]
    
    # Generate the API response
    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=messages,
        ...
    )
    
    # Retrieve the generated message
    generated_message = response['choices'][0]['message']['content']
    
    # Check if the response is empty or vague
    if not generated_message.strip():
        print("ChatGPT couldn't generate a suitable response.")
    elif "I'm not sure" in generated_message or "I don't know" in generated_message:
        print("ChatGPT is uncertain about the answer.")
    else:
    print("ChatGPT provided a response.")

Is there a better way to check if chatgpt API is answering basically "i don't know"?

Upvotes: 0

Views: 869

Answers (2)

Yuh Lee
Yuh Lee

Reputation: 353

You can check the confidence of openai's answer then decide the your one. Follow the instruction here

Upvotes: 0

DoneForAiur
DoneForAiur

Reputation: 1435

You could specify that you want ChatGPT to say "I don't know." in the role: system part and then parse the answer accordingly.

By specify I mean just like in these examples; https://platform.openai.com/examples/default-factual-answering

Upvotes: 1

Related Questions