Tool calls with open source models

[ranga-tec]

Has anyone noticed that with open-source models, even with LLaMA 3.2 70B, tool-calls are not being picked up by agents? Despite having function maps and function definitions, I've observed that agents are not suggesting some functions. Simple functions like calculators and basic database queries work fine. However, when dealing with slightly more complex functions, such as an API call with an XML string, it fails.

It works nicely with any open AI, like third-party APIs. I even tried providing function definitions within the agent prompts, but it still failed. Does anyone know how to get around this?

NASA_API_KEY = "" # Replace with your API key from api.nasa.gov

NASA_ENGINEER_PROMPT = """NASA Data Engineer with access to NASA's public APIs. Create API requests for:
BASE_URL = "https://api.nasa.gov"
API_KEY = "DEMO_KEY"

Available Endpoints:

1. APOD (Astronomy Picture of the Day)
2. Mars Rover Photos
3. Near Earth Objects (NEO)
4. Earth Observatory Natural Events

Role:

• Create NASA API requests
• Format parameters correctly
• Pass to validator

IMPORTANT: You must ALWAYS respond with a function call in this exact format:
execute_nasa_api(api_request="?", description="description of what the query does")

Example:
execute_nasa_api(api_request="/planetary/apod?date=2024-12-07", description="Get astronomy picture of the day")"""

NASA_VALIDATOR_PROMPT = """NASA Data Validator. Review and optimize API requests.

Key Tasks:

1. Validate date formats (YYYY-MM-DD)
2. Check parameter ranges and values
3. Optimize queries for response size
4. Ensure proper endpoint usage

Example Format:
INCORRECT:
/mars-photos/api/v1/rovers/curiosity/photos?sol=1000

CORRECT:
/mars-photos/api/v1/rovers/curiosity/photos?sol=1000&camera=FHAZ&page=1

Review and suggest optimizations for:

1. Date ranges
2. Data filtering
3. Pagination
4. Response format"""

NASA_ANALYST_PROMPT = """Senior NASA Data Analyst with expertise in astronomical data.

Role:

• Analyze API responses
• Extract key insights
• Identify patterns
• Generate visual reports
• Alert on significant findings

Focus Areas:

1. Astronomical events
2. Mars exploration data
3. Near-Earth objects
4. Earth observation

IMPORTANT: Execute queries and provide detailed analysis of the results."""

async def execute_nasa_api(api_request: str, description: str = None) -> Dict[str, Any]:
"""Execute NASA API request with enhanced response parsing"""
try:
api_request = api_request.strip()

    response_data = {
        'success': False,
        'result': '',
        'parsed_data': None,
        'status_code': 0,
        'execution_time': 0,
        'error': None
    }

  
    if '?' in api_request:
        api_request += f"&api_key={NASA_API_KEY}"
    else:
        api_request += f"?api_key={NASA_API_KEY}"

    base_url = "https://api.nasa.gov"
    full_url = f"{base_url}{api_request}"

    start_time = asyncio.get_event_loop().time()
    async with httpx.AsyncClient(timeout=30.0) as client:
        response = await client.get(full_url)
        response_data['status_code'] = response.status_code
        response_data['execution_time'] = asyncio.get_event_loop().time() - start_time

        if response.status_code == 200:
            response_data['result'] = response.text
            response_data['parsed_data'] = response.json()
            response_data['success'] = True
        else:
            response_data['error'] = f"HTTP Error: {response.status_code}"

    return response_data
except Exception as e:
    return {
        'success': False,
        'result': '',
        'parsed_data': None,
        'status_code': 0,
        'execution_time': 0,
        'error': str(e)
    }

config_list = {
"model": "llama3-70b-8192",
"api_key": "",
"api_type": "groq",
"functions": [
{
"name": "execute_nasa_api",
"description": "Execute NASA API requests",
"parameters": {
"type": "object",
"properties": {
"api_request": {
"type": "string",
"description": "The NASA API endpoint and parameters"
},
"description": {
"type": "string",
"description": "A description of what the API request does"
}
},
"required": ["api_request"]
}
}
]
}

nasa_engineer = autogen.AssistantAgent(
name="NASA_Engineer",
system_message=NASA_ENGINEER_PROMPT,
llm_config=config_list,
code_execution_config=False,
function_map={"execute_nasa_api": execute_nasa_api}
)

nasa_validator = autogen.AssistantAgent(
name="NASA_Validator",
system_message=NASA_VALIDATOR_PROMPT,
llm_config=config_list,
code_execution_config=False,
function_map={"execute_nasa_api": execute_nasa_api}
)

nasa_analyst = autogen.AssistantAgent(
name="NASA_Analyst",
system_message=NASA_ANALYST_PROMPT,
llm_config=config_list,
code_execution_config=False,
function_map={"execute_nasa_api": execute_nasa_api}
)

user_proxy = autogen.UserProxyAgent(
name="user_proxy",
llm_config=config_list,
human_input_mode="NEVER",
code_execution_config=False,
function_map={"execute_nasa_api": execute_nasa_api}
)

groupchat = autogen.GroupChat(
agents=[user_proxy, nasa_engineer, nasa_validator, nasa_analyst],
messages=[],
speaker_selection_method="round_robin",
max_round=10
)

manager = autogen.GroupChatManager(
groupchat=groupchat,
system_message="""Process the NASA data request by following these steps:
1. Engineer: Convert request to NASA API command
2. Validator: Check and optimize the command
3. Analyst: Execute and analyze results
When task is complete, return TERMINATE"""
)

async def start_conversation(request):
await user_proxy.a_initiate_chat(
manager,
message=f"Process this request: {request}"
)