John doe
Reputation: 21
How to Tool call with open source models in MS Autogen?
Has anyone noticed that with open-source models, even with LLaMA 3.2 70B, tool-calls are not being picked up by agents? Despite having function maps and function definitions, I've observed that agents are not suggesting some functions. Simple functions like calculators and basic database queries work fine. However, when dealing with slightly more complex functions, such as an API call with an XML string, it fails.
It works nicely with any open AI, like third-party APIs. I even tried providing function definitions within the agent prompts, but it still failed. Does anyone know how to get around this?
NASA_API_KEY = "" # Replace with your API key from api.nasa.gov
NASA_ENGINEER_PROMPT = """NASA Data Engineer with access to NASA's public APIs. Create API requests for:
BASE_URL = "https://api.nasa.gov"
API_KEY = "DEMO_KEY"
Available Endpoints:
1. APOD (Astronomy Picture of the Day)
2. Mars Rover Photos
3. Near Earth Objects (NEO)
4. Earth Observatory Natural Events
Role:
- Create NASA API requests
- Format parameters correctly
- Pass to validator
IMPORTANT: You must ALWAYS respond with a function call in this exact format:
execute_nasa_api(api_request="<endpoint>?<parameters>", description="description of what the query does")
Example:
execute_nasa_api(api_request="/planetary/apod?date=2024-12-07", description="Get astronomy picture of the day")"""
NASA_VALIDATOR_PROMPT = """NASA Data Validator. Review and optimize API requests.
Key Tasks:
1. Validate date formats (YYYY-MM-DD)
2. Check parameter ranges and values
3. Optimize queries for response size
4. Ensure proper endpoint usage
Example Format:
INCORRECT:
/mars-photos/api/v1/rovers/curiosity/photos?sol=1000
CORRECT:
/mars-photos/api/v1/rovers/curiosity/photos?sol=1000&camera=FHAZ&page=1
Review and suggest optimizations for:
1. Date ranges
2. Data filtering
3. Pagination
4. Response format"""
NASA_ANALYST_PROMPT = """Senior NASA Data Analyst with expertise in astronomical data.
Role:
- Analyze API responses
- Extract key insights
- Identify patterns
- Generate visual reports
- Alert on significant findings
Focus Areas:
1. Astronomical events
2. Mars exploration data
3. Near-Earth objects
4. Earth observation
IMPORTANT: Execute queries and provide detailed analysis of the results."""
async def execute_nasa_api(api_request: str, description: str = None) -> Dict[str, Any]:
"""Execute NASA API request with enhanced response parsing"""
try:
api_request = api_request.strip()
response_data = {
'success': False,
'result': '',
'parsed_data': None,
'status_code': 0,
'execution_time': 0,
'error': None
}
if '?' in api_request:
api_request += f"&api_key={NASA_API_KEY}"
else:
api_request += f"?api_key={NASA_API_KEY}"
base_url = "https://api.nasa.gov"
full_url = f"{base_url}{api_request}"
start_time = asyncio.get_event_loop().time()
async with httpx.AsyncClient(timeout=30.0) as client:
response = await client.get(full_url)
response_data['status_code'] = response.status_code
response_data['execution_time'] = asyncio.get_event_loop().time() - start_time
if response.status_code == 200:
response_data['result'] = response.text
response_data['parsed_data'] = response.json()
response_data['success'] = True
else:
response_data['error'] = f"HTTP Error: {response.status_code}"
return response_data
except Exception as e:
return {
'success': False,
'result': '',
'parsed_data': None,
'status_code': 0,
'execution_time': 0,
'error': str(e)
}
config_list = {
"model": "llama3-70b-8192",
"api_key": "",
"api_type": "groq",
"functions": [
{
"name": "execute_nasa_api",
"description": "Execute NASA API requests",
"parameters": {
"type": "object",
"properties": {
"api_request": {
"type": "string",
"description": "The NASA API endpoint and parameters"
},
"description": {
"type": "string",
"description": "A description of what the API request does"
}
},
"required": ["api_request"]
}
}
]
}
nasa_engineer = autogen.AssistantAgent(
name="NASA_Engineer",
system_message=NASA_ENGINEER_PROMPT,
llm_config=config_list,
code_execution_config=False,
function_map={"execute_nasa_api": execute_nasa_api}
)
nasa_validator = autogen.AssistantAgent(
name="NASA_Validator",
system_message=NASA_VALIDATOR_PROMPT,
llm_config=config_list,
code_execution_config=False,
function_map={"execute_nasa_api": execute_nasa_api}
)
nasa_analyst = autogen.AssistantAgent(
name="NASA_Analyst",
system_message=NASA_ANALYST_PROMPT,
llm_config=config_list,
code_execution_config=False,
function_map={"execute_nasa_api": execute_nasa_api}
)
user_proxy = autogen.UserProxyAgent(
name="user_proxy",
llm_config=config_list,
human_input_mode="NEVER",
code_execution_config=False,
function_map={"execute_nasa_api": execute_nasa_api}
)
groupchat = autogen.GroupChat(
agents=[user_proxy, nasa_engineer, nasa_validator, nasa_analyst],
messages=[],
speaker_selection_method="round_robin",
max_round=10
)
manager = autogen.GroupChatManager(
groupchat=groupchat,
system_message="""Process the NASA data request by following these steps:
1. Engineer: Convert request to NASA API command
2. Validator: Check and optimize the command
3. Analyst: Execute and analyze results
When task is complete, return TERMINATE"""
)
async def start_conversation(request):
await user_proxy.a_initiate_chat(
manager,
message=f"Process this request: {request}"
)
Upvotes: 0
Views: 51