LlamaIndex HuggingFaceInferenceAPI - NotImplementedError for Message Length and ValidationError for ReActAgent

Question

I’m building a ReActAgent in LlamaIndex that uses HuggingFaceInferenceAPI as the LLM. The goal is for the agent to query a SQL database in natural language using tools like NLSQLRetriever.

However, I'm encountering two main issues:

NotImplementedError: The error states “Messages passed in must be of odd length.” ValidationError: After ensuring an odd message length manually, a validation issue occurs when querying the agent.

Environment Details

LlamaIndex version: 0.12.5

Python version: 3.10

HuggingFaceInferenceAPI: google/gemma-7b-it

I’ll outline the code snippets and errors below for clarity.

Code 1: HuggingFace Inference API Setup

from llama_index.llms.huggingface import HuggingFaceInferenceAPI
import tiktoken
from llama_index.core.callbacks import CallbackManager, TokenCountingHandler
from llama_index.core import Settings
from llama_index.core.retrievers import NLSQLRetriever
from llama_index.core.query_engine import RetrieverQueryEngine
from llama_index.core.tools import QueryEngineTool, ToolMetadata
from llama_index.core.agent import ReActAgent

token_counter = TokenCountingHandler(tokenizer=tiktoken.get_encoding("cl100k_base").encode)
callback_manager = CallbackManager([token_counter])

llm = HuggingFaceInferenceAPI(
    model_name=HUGGINGFACE_MODEL_NAME,
    token=HUGGINGFACE_API_KEY,
    callback_manager=callback_manager,
    temperature=0.02,
    max_new_tokens=8000
    )
 Settings.llm = llm

 retriever = NLSQLRetriever(sql_database, tables=fetch_tables(), verbose=True)
 nl_sql_query_engine = RetrieverQueryEngine.from_args(retriever)

 query_engine_tools = [
    QueryEngineTool(
      query_engine=nl_sql_query_engine,
      metadata=ToolMetadata(
        name="retriever_query_engine",
        description="This tool allows a language model agent to interact with a SQL database."
      )
   )
 ]
 reactAgent = ReActAgent.from_tools(query_engine_tools, verbose=True, max_iterations=10)

 query = "Tell me something about Mr Fooz including number of posts he has made, posts he has answered and his badges."
 response = reactAgent.query(query)

Error 1: Odd-Length Requirement

NotImplementedError: Messages passed in must be of odd length.

Code 2: Ensuring Odd-Length Chat Messages To fix this, I attempted adding a function:

from llama_index.core.base.llms.types import ChatMessage, MessageRole
def ensure_odd_length(messages: List[ChatMessage]) -> List[ChatMessage]:
   if len(messages) % 2 == 0:
      messages.append(ChatMessage(role=MessageRole.SYSTEM, content="You are a bigquery data analyst bot"))
   return messages

 query = "Tell me something about Mr Fooz including number of posts..." 
 messages = [
    ChatMessage(role=MessageRole.SYSTEM, content="You are a bigquery data analyst bot"),
    ChatMessage(role=MessageRole.USER, content=query)
 ]
 messages = ensure_odd_length(messages)
 response = reactAgent.query(messages)

Error 2: ValidationError for ReActAgent Input

ValidationError: 2 validation errors for QueryStartEvent query.str Input should be a valid string [type=string_type, input_value=[ChatMessage(...], input_type=list] query.QueryBundle Input should be a dictionary or an instance of QueryBundle [type=dataclass_type, input_value=[ChatMessage(...)], input_type=list]

My Question

1. Why does the HuggingFace LLM in LlamaIndex require messages to have an odd length, and what’s the right approach to handle this?
2. How can I pass messages to the ReActAgent.query() method correctly to avoid the ValidationError?