Performing Function Calling with Mistral AI through Hugging Face Endpoint

Question

I am trying to perform function calling using Mistral AI through the Hugging Face endpoint. Mistral AI requires input in a specific string format (assistant: ... user: ...). However, the input format provided is not accepted as a list of messages or a prompt value (e.g., ChatPromptTemplate). When using the string format, the output only calls the function without pushing the received content from the tool call back to Mistral AI LLM.

I was trying to perform function calling using Mistral AI, which initially worked as expected. My process involved calling an external API to fetch real-time data and then attempting to push this data back to the LLM (Language Model) using messages with the tool call name and ID. I expected the final output to combine my data with the LLM's response, resulting in a cohesive answer. However, instead of getting the expected response, the system kept returning a function call as the response again.

# Import necessary libraries
from langchain_huggingface.chat_models import ChatHuggingFace
from langchain.prompts import PromptTemplate
from langchain.chains import RetrievalQA
from langchain.document_loaders import TextLoader
from langchain.llms import HuggingFacePipeline
from langchain_text_splitters.character import CharacterTextSplitter
from langchain.prompts import PromptTemplate
from langchain_core.pydantic_v1 import BaseModel, Field
from langchain_core.output_parsers.openai_tools import PydanticToolsParser
from langchain_huggingface import HuggingFaceEndpoint
from langchain_core.utils.function_calling import convert_to_openai_function
from langchain.prompts import PromptTemplate, ChatPromptTemplate
from langchain_core.messages import (
    HumanMessage,
    SystemMessage,
    AIMessage,
    ChatMessage
)
from dotenv import load_dotenv, find_dotenv
import os
import serpapi
import serpapi.client




# Load environment variables
_ = load_dotenv(find_dotenv())
serp_key = os.getenv('SERP_API_KEY')




# Configure the LLM endpoint
llm = HuggingFaceEndpoint(
    repo_id='mistralai/Mistral-7B-Instruct-v0.3',
    huggingfacehub_api_token='*********',
    task="text-generation",
    model_kwargs={
        "min_length": 200,
        "max_length": 2000,
        "num_return_sequences": 1
    },
    temperature=0.5
)



# Define prompt and functions
prompt_str = "I want you to act as travel organizer, Take input from user extract the necessary details plan the trip route using details you need to plan the route and time going to be spent"
functions = [
    {
        "name": "plan_holiday",
        "description": "Plan a holiday based on user's interests",
        "parameters": {
            "type": "object",
            "properties": {
                "destination": {
                    "type": "string",
                    "description": "The destination of the holiday",
                },
                "duration": {
                    "type": "integer",
                    "description": "The duration of the trip in holiday",
                },
            },
            "required": ["destination", "duration"],
        },
    }
]

# Generate messages
messages = [
    SystemMessage(content=prompt_str),
    HumanMessage(
        content="I am thinking of having a 12 day long vacation in Venice, can you help me plan it?"
    ),
]

# Format prompt
chat_pr = ChatPromptTemplate(messages=messages).format()
print(chat_pr)

# Create ChatHuggingFace model
chat_model = ChatHuggingFace(llm=llm)
functions = [convert_to_openai_function(i) for i in functions]

# Bind tools to the model
llm_with_holiday_planning = chat_model.bind_tools(functions, tool_choice='auto')

# Invoke the model with the formatted prompt
response = llm_with_holiday_planning.invoke(chat_pr)
print(response)

# Append response to messages
messages.append(ChatMessage(role='assistant', content='', tool_calls=response.additional_kwargs['tool_calls'], id=response.additional_kwargs['tool_calls'][0]['id']))

# Function to search Google for a place
def search_google_for(place_search):
    sero_cj = serpapi.Client(api_key=serp_key)
    result = sero_cj.search(
        params={
            "q": "Venice tourist places",
            "location": "India",
            "hl": "en",
            "gl": "In",
            "google_domain": "google.com",
        }
    )
    return result

# Fetch results using the function
results = search_google_for(response.additional_kwargs['tool_calls'][0].function['arguments'])
print(response.additional_kwargs['tool_calls'][0])

# Extract and compile information about places
info_about_places = ""
if results.get("organic_results"):
    for result in results["organic_results"]:
        title = result.get("title")
        snippet = result.get("snippet")
        link = result.get("link")
        info_about_places += snippet

# Append function result to messages
messages.append(ChatMessage(role='function', name=response.additional_kwargs['tool_calls'][0].function['name'], content=info_about_places, tool_call_id=response.additional_kwargs['tool_calls'][0]['id']))

# Format and invoke the updated prompt
chat_pr = ChatPromptTemplate(messages=messages).format()
print(chat_pr)
print(llm_with_holiday_planning.invoke(chat_pr))

Setting Up the Environment:

Loaded necessary libraries and environment variables. Configured the Mistral AI model endpoint using the Hugging Face API. Defining the Prompt and Functions:

Created a prompt instructing the model to act as a travel organizer. Defined functions such as plan_holiday to handle specific tasks. Generating Messages:

Constructed messages using SystemMessage and HumanMessage to simulate a conversation. Binding Tools to the Model:

Converted the defined functions to an OpenAI-compatible format. Bound these functions to the chat model using bind_tools. Invoking the Model:

Attempted to invoke the model with the formatted prompt and expected it to call the defined function and use the external API data. Expected Outcome: I expected the model to process the function call, fetch data from the external API, and integrate this data into a coherent response along with the LLM's generated content. Actual Result: Instead of providing a combined response with both the fetched data and the LLM's generated content, the model repeatedly returned a function call as the response. your text

Performing Function Calling with Mistral AI through Hugging Face Endpoint

Answers (1)

Related Questions