A4F

Documentation

Features

Tool & Function Calling

Use tools in your prompts with A4F API Services.

Tool calls (also known as function calls) give an LLM access to external tools. The LLM does not call the tools directly. Instead, it suggests one or more tools to call based on the user's prompt and the provided tool definitions. The user's application code then calls the tool separately and provides the results back to the LLM. Finally, the LLM uses these results to formulate a response to the user's original question.

A4F API Services standardizes the tool calling interface across models and providers that support this feature. Since A4F aims for OpenAI API compatibility, you can leverage the familiar OpenAI SDK structure for tool calling. Note that tool/function calling capabilities depend on the specific model and the underlying A4F provider. Provider-3 is specifically mentioned as supporting function calling.

For a primer on how tool calling works in the OpenAI SDK, please see this article from OpenAI, or if you prefer to learn from a full end-to-end example using A4F, keep reading.

Tool Calling Example

Here is Python code that gives LLMs the ability to call an external API — in this case Project Gutenberg, to search for books.

First, let's do some basic setup:

from openai import OpenAI


MODEL = "provider-3/openai/gpt-4o"  and model known for tool calling


a4f_client = OpenAI(
  base_url="https://api.a4f.co/v1",
  api_key="YOUR_A4F_API_KEY", # Replace with your actual A4F API key
)


task = "What are the titles of some James Joyce books?"


messages = [
    {
        "role": "system",
        "content": "You are a helpful assistant."
    },
    {
        "role": "user",
        "content": task,
    }
]

Define the Tool

Next, we define the tool that we want to call. Remember, the tool is going to get requestedby the LLM, but the code we are writing here (your application code) is ultimately responsible for executing the tool/function call and returning the results to the LLM.

import requests
import json


def search_gutenberg_books(search_terms: str):
    """Search for books on Project Gutenberg based on search terms."""
    search_query = " ".join(search_terms)
    url = f"https://gutendex.com/books"
    response = requests.get(url, params={"search": search_query})
    response.raise_for_status()
    
    simplified_results = []
    for book in response.json().get("results", []):
        simplified_results.append({
            "id": book.get("id"),
            "title": book.get("title"),
            "authors": [author.get("name") for author in book.get("authors", [])], # Extract author names
        })
    return simplified_results


tools = [
  {
    "type": "function",
    "function": {
      "name": "search_gutenberg_books",
      "description": "Search for books on Project Gutenberg based on search terms. Returns a list of books with their titles and authors.",
      "parameters": {
        "type": "object",
        "properties": {
          "search_terms": {
            "type": "string",
            "description": "The search terms to look for, e.g. 'Leo Tolstoy great gatsby'"
          }
        },
        "required": ["search_terms"]
      }
    }
  }
]


TOOL_MAPPING = {
    "search_gutenberg_books": search_gutenberg_books,
}

Note that the "tool" itself is just a normal Python function (`search_gutenberg_books`). We then write a JSON "spec" (the `tools` list) compatible with the OpenAI function calling parameter. We'll pass this spec to the LLM via the A4F API so that it knows this tool is available and how to use it (i.e., what arguments it expects). The LLM will request the tool when needed, along with any arguments it deems necessary. We'll then execute the tool call locally within our application, make the actual function call, and return the results to the LLM.

Tool use and tool results

Let's make the first A4F API call to the model, providing the tools definition:

request_1 = {
    "model": MODEL,
    "tools": tools,
    "messages": messages
}


response_1_completion = a4f_client.chat.completions.create(**request_1)
response_1_message = response_1_completion.choices[0].message


messages.append(response_1_message)


if response_1_message.tool_calls:
    for tool_call in response_1_message.tool_calls:
        tool_name = tool_call.function.name
        tool_args_str = tool_call.function.arguments
        try:
            tool_args = json.loads(tool_args_str)
        except json.JSONDecodeError:
            print(f"Error: Could not decode tool arguments: {tool_args_str}")
            continue


        print(f"--- Calling tool: {tool_name} with args: {tool_args} ---")
        
        if tool_name in TOOL_MAPPING:
            function_to_call = TOOL_MAPPING[tool_name]
            try:
                tool_response = function_to_call(search_terms=tool_args.get("search_terms"))
            except Exception as e:
                print(f"Error calling tool {tool_name}: {e}")
                tool_response = {"error": str(e)}
        else:
            print(f"Error: Tool '{tool_name}' not found in TOOL_MAPPING.")
            tool_response = {"error": f"Tool '{tool_name}' not found."}


        print(f"--- Tool Response: {tool_response} ---")


        messages.append({
            "role": "tool",
            "tool_call_id": tool_call.id,
            "name": tool_name,
            "content": json.dumps(tool_response), 
        })
else:
    print("No tool call requested by the model.")

The LLM responds with a finish_reason of tool_calls, and a tool_calls array in its message object. In a generic LLM response-handler, you would want to check the finish_reason before processing tool calls, but here we will assume it's the case. Let's keep going, by processing the tool call:

The messages array now has:

  1. Our original system and user messages.
  2. The LLM's response message (containing the tool call request).
  3. The result of our tool execution (formatted as a "tool" role message).

Now, we can make a second A4F API call, sending the updated messages list back to the model, and hopefully get our final result!

request_2 = {
    "model": MODEL,
    "messages": messages, 
    "tools": tools 
}


response_2_completion = a4f_client.chat.completions.create(**request_2)
final_message_content = response_2_completion.choices[0].message.content
print(final_message_content)

The output will be something like:

Here are some books by James Joyce:
* "Ulysses"
* "Dubliners"
* "A Portrait of the Artist as a Young Man"
* "Chamber Music"
* "Exiles: A Play in Three Acts"

We did it! We've successfully used a tool in a prompt via A4F API Services.

A Simple Agentic Loop

In the example above, the calls are made explicitly and sequentially. To handle a wide variety of user inputs and potentially multiple tool calls, or a sequence of tool calls, you can use an agentic loop.

Here's an example of a simple agentic loop (using the same tools definition and TOOL_MAPPING as above):

def call_llm(msgs):
    resp_completion = a4f_client.chat.completions.create(
        model=MODEL, 
        tools=tools, 
        messages=msgs
    )
    msgs.append(resp_completion.choices[0].message) 
    return resp_completion 


def get_tool_response_from_llm_response(llm_completion_response):
    if not llm_completion_response.choices[0].message.tool_calls:
        return None 


    tool_call = llm_completion_response.choices[0].message.tool_calls[0]
    tool_name = tool_call.function.name
    tool_args_str = tool_call.function.arguments
    try:
        tool_args = json.loads(tool_args_str)
    except json.JSONDecodeError:
        print(f"Error decoding arguments for {tool_name}: {tool_args_str}")
        return {
            "role": "tool",
            "tool_call_id": tool_call.id,
            "name": tool_name,
            "content": json.dumps({"error": "Invalid arguments format received from LLM."})
        }


    if tool_name in TOOL_MAPPING:
        function_to_call = TOOL_MAPPING[tool_name]
        try:
            tool_result = function_to_call(**tool_args) 
        except Exception as e:
            print(f"Error executing tool {tool_name}: {e}")
            tool_result = {"error": f"Failed to execute tool {tool_name}: {str(e)}"}
    else:
        print(f"Tool {tool_name} not found in mapping.")
        tool_result = {"error": f"Tool {tool_name} not recognized."}


    return {
        "role": "tool",
        "tool_call_id": tool_call.id,
        "name": tool_name,
        "content": json.dumps(tool_result)
    }


current_messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What books did James Joyce write?"}
]


MAX_ITERATIONS = 5
for i in range(MAX_ITERATIONS):
    print(f"--- Iteration: {i + 1} ---")
    llm_response_completion = call_llm(current_messages) 
    
    if llm_response_completion.choices[0].message.tool_calls:
        print("LLM requested tool call(s).")
        tool_message = get_tool_response_from_llm_response(llm_response_completion)
        if tool_message:
            current_messages.append(tool_message)
    else:
        print("LLM finished, no tool calls requested.")
        final_answer = llm_response_completion.choices[0].message.content
        print(f"\nFinal Answer:\n{final_answer}")
        break 
else:
    print("Max iterations reached without a final answer.")

Was this page helpful?

Provider Routing

Route requests to the best provider for your needs.

Previous

Vision & PDFs

Process images and understand PDF handling with A4F.

Next