November 21, 2026

Gemini API tool retrieval: cut tool tokens in Python

Build a 30-tool Gemini agent in Python, defer rarely used tools with tool_search, and compare total_input_tokens before and after.

Guide

Tech

On the Gemini API's Interactions API, you set defer_loading: true on the tools you rarely use and add {"type": "tool_search"} to tools. Gemini then loads a schema only when it needs one. Keep your most used tools loaded. With fewer than 20 stable tools, Google's own mode table puts you on static loading, because searches cost tokens too.

You'll build an ops desk agent with 30 tools, run six fixed prompts with and without deferral, and print a table of tokens from your own key. Tool retrieval is in preview and needs the v1beta API version, so the surface can change before general availability.

What you'll build

A Python command-line agent with a 30-function tool library: 4 core tools that always load and 26 that are deferred. The tool names, schemas and stub implementations are fixtures written for this tutorial, not Google's. measure.py runs the same six prompts in two modes, static (every schema sent) and deferred, and prints input tokens, tool-use tokens, their sum, and whether the right tool was called. Plan on about 45 minutes.

The project layout:

text
ops-desk-agent/
├── .env                # your key, git-ignored
├── .env.example
├── .gitignore
├── agent.py            # one prompt through the Interactions API
├── measure.py          # runs both modes and prints the table
├── prompts.json        # six fixed prompts
└── tools_catalog.py    # the 30 tools and their stub implementations

Here's the shape of the finished output. These numbers are placeholders to show the layout, not a result: your run prints its own, and the last two lines show how the dollar figure is worked out.

text
mode      input  tool_use     sum  right  prompt
static     3000         0    3000  yes    Is the payments service up?
deferred    900         0     900  yes    Is the payments service up?
static     3000         0    3000  yes    Roll back the last release of the checko
deferred    900      1100    2000  yes    Roll back the last release of the checko
static: 18000 tokens, about $27.00 per 1,000 runs of this set
deferred: 9800 tokens, about $14.70 per 1,000 runs of this set

What you need before you start

  • Python 3 with venv and pip. The Interactions API needs google-genai 2.3.0 or later, and the install lines pin it.
  • A macOS or Linux shell. On Windows, use WSL.
  • A Gemini API key from the API keys page in Google AI Studio, stored in the GEMINI_API_KEY environment variable. The pricing page lists a free tier for gemini-4-hydrogen, the model used here, so a free key is enough to follow along.

measure.py sends about two dozen requests per run, because each prompt makes two calls in each mode. Keep that in mind if your key has a low rate limit.

Quickstart

Already have a key? Install the SDK, export the key, and run this:

Bash
pip install -U "google-genai>=2.3.0"
export GEMINI_API_KEY="your-api-key-here"
Python
from google import genai

client = genai.Client(http_options={"api_version": "v1beta"})

tools = [
    {"type": "tool_search"},
    {
        "type": "function",
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string", "description": "The city name."}},
            "required": ["city"],
        },
        "defer_loading": True,
    },
    {
        "type": "function",
        "name": "get_stock_price",
        "description": "Get the latest price for a stock ticker.",
        "parameters": {
            "type": "object",
            "properties": {"ticker": {"type": "string", "description": "The ticker."}},
            "required": ["ticker"],
        },
        "defer_loading": True,
    },
]

interaction = client.interactions.create(
    model="gemini-4-hydrogen",
    input="What's the weather in Paris?",
    tools=tools,
)

for step in interaction.steps:
    print(step.type, getattr(step, "name", ""))

Three things to get right before you build on it:

  • Put {"type": "tool_search"} in tools on every request. Without it, defer_loading returns an error.
  • Send the full tools array again on every request, including the follow-up that returns a tool result.
  • Don't defer a tool your system instructions name. Keep it loaded.

How to build the agent and measure the saving

1. Create the project and install everything

Make the folder, a virtual environment, one install line for every package the tutorial uses, your key file, and the prompt fixture. Everything later builds on this folder.

In a terminal, from wherever you keep projects:

Bash
mkdir ops-desk-agent && cd ops-desk-agent
python3 -m venv .venv && source .venv/bin/activate
pip install -U "google-genai>=2.3.0" python-dotenv

Create .env.example, a template that's safe to commit:

text
GEMINI_API_KEY=

Copy it to .env and paste your real key after the equals sign:

Bash
cp .env.example .env

Create .gitignore:

text
.env
.venv/
__pycache__/

Create prompts.json. Four prompts need a deferred tool and two need a core tool. expect is the function the model should call:

JSON
[
  {"prompt": "Is the payments service up?", "expect": "get_service_status"},
  {"prompt": "Who is on call for the platform team?", "expect": "get_oncall"},
  {"prompt": "Roll back the last release of the checkout service.", "expect": "deploy_rollback_release"},
  {"prompt": "Show me the plan for customer cus_1042.", "expect": "billing_lookup_customer_plan"},
  {"prompt": "Purge the CDN cache for /static/app.js.", "expect": "cdn_purge_cache"},
  {"prompt": "How many connections does the orders database have right now?", "expect": "db_get_connection_count"}
]

The SDK reads GEMINI_API_KEY from the environment but doesn't read .env files, so each Python file here calls load_dotenv() from python-dotenv first.

Checkpoint, in the same terminal:

Bash
pip show google-genai

You should see something like (the version will differ):

text
Name: google-genai
Version: <installed version>

If your prompt doesn't start with (.venv), re-run the source .venv/bin/activate line.

2. Generate the 30-tool library

Create tools_catalog.py. It holds one compact spec per tool and builds the full function schemas from them, so you don't paste 30 hand-written schemas. The fourth value in each spec marks the 4 core tools that step 5 keeps loaded. This file is tutorial glue, not Google code.

Python
import json

SPECS = [
    ("get_service_status", "Get the current status of a service.", {"service": "string"}, True),
    ("get_oncall", "Get who is on call for a team.", {"team": "string"}, True),
    ("create_ticket", "Create a ticket in the tracker.", {"title": "string"}, True),
    ("search_runbook", "Search the runbooks by keyword.", {"query": "string"}, True),
    ("db_list_backups", "List backups for a database.", {"database": "string"}, False),
    ("db_restore_backup", "Restore a database from a backup.", {"database": "string", "backup_id": "string"}, False),
    ("db_get_connection_count", "Get the open connection count for a database.", {"database": "string"}, False),
    ("db_run_vacuum", "Run a vacuum on a database.", {"database": "string"}, False),
    ("k8s_restart_worker_pool", "Restart a Kubernetes worker pool.", {"pool": "string"}, False),
    ("k8s_scale_deployment", "Scale a Kubernetes deployment.", {"deployment": "string", "replicas": "integer"}, False),
    ("k8s_get_pod_logs", "Get recent logs for a pod.", {"pod": "string"}, False),
    ("k8s_list_namespaces", "List the namespaces in a cluster.", {"cluster": "string"}, False),
    ("billing_lookup_customer_plan", "Look up the plan a customer is on.", {"customer_id": "string"}, False),
    ("billing_issue_refund", "Issue a refund for an invoice.", {"invoice_id": "string"}, False),
    ("billing_list_invoices", "List the invoices for a customer.", {"customer_id": "string"}, False),
    ("billing_apply_credit", "Apply an account credit to a customer.", {"customer_id": "string", "amount": "integer"}, False),
    ("iam_rotate_api_key", "Rotate the API key of a service.", {"service": "string"}, False),
    ("iam_list_users", "List the users on a team.", {"team": "string"}, False),
    ("iam_disable_user", "Disable a user account.", {"user": "string"}, False),
    ("iam_grant_role", "Grant a role to a user.", {"user": "string", "role": "string"}, False),
    ("dns_get_record", "Get the DNS record for a hostname.", {"hostname": "string"}, False),
    ("dns_update_record", "Update the DNS record for a hostname.", {"hostname": "string", "value": "string"}, False),
    ("cdn_purge_cache", "Purge a path from the CDN cache.", {"path": "string"}, False),
    ("cdn_get_hit_rate", "Get the CDN cache hit rate for a domain.", {"domain": "string"}, False),
    ("alerts_silence_alert", "Silence an alert for some minutes.", {"alert_id": "string", "minutes": "integer"}, False),
    ("alerts_list_firing", "List the alerts firing for a team.", {"team": "string"}, False),
    ("deploy_get_last_release", "Get the last release of a service.", {"service": "string"}, False),
    ("deploy_rollback_release", "Roll back the last release of a service.", {"service": "string"}, False),
    ("cost_get_monthly_spend", "Get the monthly cloud spend for an account.", {"account": "string"}, False),
    ("cost_list_idle_instances", "List idle instances in an account.", {"account": "string"}, False),
]


def _build(name, description, params, defer):
    tool = {
        "type": "function",
        "name": name,
        "description": description,
        "parameters": {
            "type": "object",
            "properties": {
                p: {"type": t, "description": "The " + p.replace("_", " ") + "."}
                for p, t in params.items()
            },
            "required": list(params),
        },
    }
    if defer:
        tool["defer_loading"] = True
    return tool


def static_tools():
    return [_build(n, d, p, False) for n, d, p, _core in SPECS]


def run_tool(name, args):
    return json.dumps({"tool": name, "args": dict(args or {}), "status": "ok"})


if __name__ == "__main__":
    tools = static_tools()
    print("tools:", len(tools))
    print("schema characters:", len(json.dumps(tools)))

static_tools() returns every tool with its full parameters schema. run_tool is a stub that echoes the call back as JSON, so the agent has something to execute.

Checkpoint, from ops-desk-agent/:

Bash
python tools_catalog.py

You should see something like (your character count will differ):

text
tools: 30
schema characters: <number>

That character count is the weight static loading puts on every request.

3. Send every schema on every request

Create agent.py. It sends one prompt with all 30 schemas attached, which is the baseline you'll measure deferral against. The client is pinned to v1beta because the Interactions API's tool retrieval needs it.

Python
from dotenv import load_dotenv
from google import genai

from tools_catalog import static_tools

load_dotenv()

MODEL = "gemini-4-hydrogen"

client = genai.Client(http_options={"api_version": "v1beta"})

if __name__ == "__main__":
    interaction = client.interactions.create(
        model=MODEL,
        input="Is the payments service up?",
        tools=static_tools(),
    )
    for step in interaction.steps:
        print(step.type, getattr(step, "name", ""))
    print("total_input_tokens:", interaction.usage.total_input_tokens)

Checkpoint:

Bash
python agent.py

You should see something like (the token count will differ):

text
function_call get_service_status
total_input_tokens: <number>

The model asked you to call get_service_status instead of answering. If you see an authentication error, check that .env sits in this folder and holds your real key.

4. Run the tool and return the result

Replace the whole of agent.py. Now ask() runs the loop: it executes each function_call, sends the result back with previous_interaction_id, and adds up the usage of every request. It checks the steps rather than status, because Google's guide says status isn't always requires_action.

Python
import sys

from dotenv import load_dotenv
from google import genai

from tools_catalog import run_tool, static_tools

load_dotenv()

MODEL = "gemini-4-hydrogen"
MAX_TURNS = 5

client = genai.Client(http_options={"api_version": "v1beta"})


def ask(prompt, tools):
    interaction = client.interactions.create(model=MODEL, input=prompt, tools=tools)
    step_types = []
    called = []
    tokens_in = 0
    tokens_tool = 0
    for _ in range(MAX_TURNS):
        usage = interaction.usage
        tokens_in += usage.total_input_tokens or 0
        tokens_tool += getattr(usage, "total_tool_use_tokens", None) or 0
        step_types += [step.type for step in interaction.steps]
        calls = [step for step in interaction.steps if step.type == "function_call"]
        if not calls:
            break
        results = []
        for call in calls:
            called.append(call.name)
            results.append(
                {
                    "type": "function_result",
                    "name": call.name,
                    "call_id": call.id,
                    "result": [{"type": "text", "text": run_tool(call.name, call.arguments)}],
                }
            )
        interaction = client.interactions.create(
            model=MODEL,
            input=results,
            previous_interaction_id=interaction.id,
            tools=tools,
        )
    return {
        "text": interaction.output_text or "",
        "steps": step_types,
        "called": called,
        "input": tokens_in,
        "tool_use": tokens_tool,
    }


if __name__ == "__main__":
    result = ask(sys.argv[1], static_tools())
    print(result["text"])

tools=tools goes into the follow-up request too, as the Quickstart said. Both usage fields are optional, so each is added with or 0 and a request that ran no search still works.

Checkpoint:

Bash
python agent.py "Is the payments service up?"

You should see something like (the wording will differ):

text
The payments service reports status ok.

If the output is empty, print result["steps"] to see where the loop stopped.

5. Defer the long tail

Add deferral to tools_catalog.py, then let agent.py pick a mode. The 26 non-core tools get defer_loading: True, the 4 core tools stay loaded, and {"type": "tool_search"} goes in the array. Gemini then sees only each deferred tool's name and description until it asks for a schema.

Append this to the end of tools_catalog.py, below the if __name__ == "__main__": block:

Python
def deferred_tools():
    tools = [{"type": "tool_search"}]
    tools += [_build(n, d, p, not core) for n, d, p, core in SPECS]
    return tools

In agent.py, replace the import line from tools_catalog import run_tool, static_tools with:

Python
from tools_catalog import deferred_tools, run_tool, static_tools

Then replace the if __name__ == "__main__": block at the bottom of agent.py with:

Python
if __name__ == "__main__":
    tools = deferred_tools() if sys.argv[1] == "deferred" else static_tools()
    result = ask(sys.argv[2], tools)
    print("steps:", " > ".join(result["steps"]))
    print("called:", result["called"])
    print("text:", result["text"])
    print("total_input_tokens:", result["input"], "total_tool_use_tokens:", result["tool_use"])

Checkpoint:

Bash
python agent.py deferred "Roll back the last release of the checkout service."

You should see something like (the wording and counts will differ):

text
steps: tool_search_call > tool_search_result > function_call > model_output
called: ['deploy_rollback_release']
text: The checkout service was rolled back.
total_input_tokens: <number> total_tool_use_tokens: <number>

The first three step types are the ones to look for: a search, its result, then the real call. Searches run inside the same API call and previous_interaction_id keeps the history on Google's side, so you never handle or resend tool_search_call yourself.

6. Compare both modes on the same prompts

Create measure.py. It runs every prompt in prompts.json in both modes and prints one row per run, plus a total and a dollar estimate per mode.

Each server-side search triggers an extra model call with the full context plus the loaded schemas. Those tokens land in total_tool_use_tokens, billed at the input rate, so the sum is the figure that matters. With a small library, or a long context that searches every turn, static loading can win.

Hydrogen's paid input price is $1.50 per million tokens on the Standard tier, which is the constant in the file. Check Google's pricing page before you trust the dollar line.

Python
import json

from agent import ask
from tools_catalog import deferred_tools, static_tools

PRICE_PER_MILLION_INPUT = 1.50
MODES = {"static": static_tools, "deferred": deferred_tools}


def main():
    with open("prompts.json") as f:
        prompts = json.load(f)
    totals = {mode: 0 for mode in MODES}
    print("mode      input  tool_use     sum  right  prompt")
    for item in prompts:
        for mode, build in MODES.items():
            result = ask(item["prompt"], build())
            total = result["input"] + result["tool_use"]
            totals[mode] += total
            right = "yes" if item["expect"] in result["called"] else "NO"
            print(
                f"{mode:<9} {result['input']:>6} {result['tool_use']:>9} {total:>7}  "
                f"{right:<5}  {item['prompt'][:40]}"
            )
    for mode, total in totals.items():
        dollars = total / 1_000_000 * PRICE_PER_MILLION_INPUT * 1000
        print(f"{mode}: {total} tokens, about ${dollars:.2f} per 1,000 runs of this set")


if __name__ == "__main__":
    main()

Checkpoint:

Bash
python measure.py

You should see 12 rows and two total lines in the layout from What you'll build, with your own numbers.

7. Run it end to end and check the edge cases

Read the table before you trust it. Three things tell you whether deferral earned its place on your prompts: sum per row, right per row, and the totals.

First, look at the two core-tool prompts (the first two in prompts.json). Run one in deferred mode and confirm the steps hold no tool_search_call:

Bash
python agent.py deferred "Is the payments service up?"

You should see something like:

text
steps: function_call > model_output
called: ['get_service_status']
text: The payments service reports status ok.
total_input_tokens: <number> total_tool_use_tokens: 0

A core tool needs no search, so deferred mode should pay nothing for one here. Then run a prompt that matches no tool:

Bash
python agent.py deferred "Book a meeting room for Friday."

You should see something like (the wording will differ):

text
steps: tool_search_call > tool_search_result > model_output
called: []
text: I don't have a tool for booking rooms.
total_input_tokens: <number> total_tool_use_tokens: <number>

A search that matches no deferred tool gives you a text response, not a structured error, so handle called == [] as a normal outcome. Finally, scan the right column. Google publishes no accuracy figure for retrieval, so a NO in deferred mode where static said yes is the finding to chase, and your own prompts are the only place to find it.

Your files now match the ones in Full code.

Search more than 300,000 open roles in one place
From more than 19,000 companies across the UK, the US and Spain. Free during the open beta, until the end of November.
Try Job Search

How the pieces fit

text
your CLI (agent.py, measure.py)
  |  tools = core schemas + deferred names/descriptions + tool_search
  |  GEMINI_API_KEY (read from .env, never sent to a browser)
  v
Interactions API (v1beta, gemini-4-hydrogen)
  |  tool_search_call > tool_search_result > function_call
  |  (searches resolve inside one API call)
  v
your tool functions (run_tool in tools_catalog.py)
  |  function_result sent back with previous_interaction_id
  v
final output_text

Clean up

Leave the virtual environment:

Bash
deactivate

To remove the project, delete the ops-desk-agent folder, which also deletes .env. If you made the API key only for this tutorial, revoke it in Google AI Studio.

Full code

Steps 1 to 6 build every file below. The project layout is in What you'll build.

Install and run, from ops-desk-agent/ with the virtual environment from step 1 active:

Bash
pip install -U "google-genai>=2.3.0" python-dotenv
cp .env.example .env   # then put your real key in .env
python measure.py

.env.example:

text
GEMINI_API_KEY=

.gitignore:

text
.env
.venv/
__pycache__/

prompts.json:

JSON
[
  {"prompt": "Is the payments service up?", "expect": "get_service_status"},
  {"prompt": "Who is on call for the platform team?", "expect": "get_oncall"},
  {"prompt": "Roll back the last release of the checkout service.", "expect": "deploy_rollback_release"},
  {"prompt": "Show me the plan for customer cus_1042.", "expect": "billing_lookup_customer_plan"},
  {"prompt": "Purge the CDN cache for /static/app.js.", "expect": "cdn_purge_cache"},
  {"prompt": "How many connections does the orders database have right now?", "expect": "db_get_connection_count"}
]

tools_catalog.py:

Python
import json

SPECS = [
    ("get_service_status", "Get the current status of a service.", {"service": "string"}, True),
    ("get_oncall", "Get who is on call for a team.", {"team": "string"}, True),
    ("create_ticket", "Create a ticket in the tracker.", {"title": "string"}, True),
    ("search_runbook", "Search the runbooks by keyword.", {"query": "string"}, True),
    ("db_list_backups", "List backups for a database.", {"database": "string"}, False),
    ("db_restore_backup", "Restore a database from a backup.", {"database": "string", "backup_id": "string"}, False),
    ("db_get_connection_count", "Get the open connection count for a database.", {"database": "string"}, False),
    ("db_run_vacuum", "Run a vacuum on a database.", {"database": "string"}, False),
    ("k8s_restart_worker_pool", "Restart a Kubernetes worker pool.", {"pool": "string"}, False),
    ("k8s_scale_deployment", "Scale a Kubernetes deployment.", {"deployment": "string", "replicas": "integer"}, False),
    ("k8s_get_pod_logs", "Get recent logs for a pod.", {"pod": "string"}, False),
    ("k8s_list_namespaces", "List the namespaces in a cluster.", {"cluster": "string"}, False),
    ("billing_lookup_customer_plan", "Look up the plan a customer is on.", {"customer_id": "string"}, False),
    ("billing_issue_refund", "Issue a refund for an invoice.", {"invoice_id": "string"}, False),
    ("billing_list_invoices", "List the invoices for a customer.", {"customer_id": "string"}, False),
    ("billing_apply_credit", "Apply an account credit to a customer.", {"customer_id": "string", "amount": "integer"}, False),
    ("iam_rotate_api_key", "Rotate the API key of a service.", {"service": "string"}, False),
    ("iam_list_users", "List the users on a team.", {"team": "string"}, False),
    ("iam_disable_user", "Disable a user account.", {"user": "string"}, False),
    ("iam_grant_role", "Grant a role to a user.", {"user": "string", "role": "string"}, False),
    ("dns_get_record", "Get the DNS record for a hostname.", {"hostname": "string"}, False),
    ("dns_update_record", "Update the DNS record for a hostname.", {"hostname": "string", "value": "string"}, False),
    ("cdn_purge_cache", "Purge a path from the CDN cache.", {"path": "string"}, False),
    ("cdn_get_hit_rate", "Get the CDN cache hit rate for a domain.", {"domain": "string"}, False),
    ("alerts_silence_alert", "Silence an alert for some minutes.", {"alert_id": "string", "minutes": "integer"}, False),
    ("alerts_list_firing", "List the alerts firing for a team.", {"team": "string"}, False),
    ("deploy_get_last_release", "Get the last release of a service.", {"service": "string"}, False),
    ("deploy_rollback_release", "Roll back the last release of a service.", {"service": "string"}, False),
    ("cost_get_monthly_spend", "Get the monthly cloud spend for an account.", {"account": "string"}, False),
    ("cost_list_idle_instances", "List idle instances in an account.", {"account": "string"}, False),
]


def _build(name, description, params, defer):
    tool = {
        "type": "function",
        "name": name,
        "description": description,
        "parameters": {
            "type": "object",
            "properties": {
                p: {"type": t, "description": "The " + p.replace("_", " ") + "."}
                for p, t in params.items()
            },
            "required": list(params),
        },
    }
    if defer:
        tool["defer_loading"] = True
    return tool


def static_tools():
    return [_build(n, d, p, False) for n, d, p, _core in SPECS]


def run_tool(name, args):
    return json.dumps({"tool": name, "args": dict(args or {}), "status": "ok"})


if __name__ == "__main__":
    tools = static_tools()
    print("tools:", len(tools))
    print("schema characters:", len(json.dumps(tools)))


def deferred_tools():
    tools = [{"type": "tool_search"}]
    tools += [_build(n, d, p, not core) for n, d, p, core in SPECS]
    return tools

agent.py:

Python
import sys

from dotenv import load_dotenv
from google import genai

from tools_catalog import deferred_tools, run_tool, static_tools

load_dotenv()

MODEL = "gemini-4-hydrogen"
MAX_TURNS = 5

client = genai.Client(http_options={"api_version": "v1beta"})


def ask(prompt, tools):
    interaction = client.interactions.create(model=MODEL, input=prompt, tools=tools)
    step_types = []
    called = []
    tokens_in = 0
    tokens_tool = 0
    for _ in range(MAX_TURNS):
        usage = interaction.usage
        tokens_in += usage.total_input_tokens or 0
        tokens_tool += getattr(usage, "total_tool_use_tokens", None) or 0
        step_types += [step.type for step in interaction.steps]
        calls = [step for step in interaction.steps if step.type == "function_call"]
        if not calls:
            break
        results = []
        for call in calls:
            called.append(call.name)
            results.append(
                {
                    "type": "function_result",
                    "name": call.name,
                    "call_id": call.id,
                    "result": [{"type": "text", "text": run_tool(call.name, call.arguments)}],
                }
            )
        interaction = client.interactions.create(
            model=MODEL,
            input=results,
            previous_interaction_id=interaction.id,
            tools=tools,
        )
    return {
        "text": interaction.output_text or "",
        "steps": step_types,
        "called": called,
        "input": tokens_in,
        "tool_use": tokens_tool,
    }


if __name__ == "__main__":
    tools = deferred_tools() if sys.argv[1] == "deferred" else static_tools()
    result = ask(sys.argv[2], tools)
    print("steps:", " > ".join(result["steps"]))
    print("called:", result["called"])
    print("text:", result["text"])
    print("total_input_tokens:", result["input"], "total_tool_use_tokens:", result["tool_use"])

measure.py:

Python
import json

from agent import ask
from tools_catalog import deferred_tools, static_tools

PRICE_PER_MILLION_INPUT = 1.50
MODES = {"static": static_tools, "deferred": deferred_tools}


def main():
    with open("prompts.json") as f:
        prompts = json.load(f)
    totals = {mode: 0 for mode in MODES}
    print("mode      input  tool_use     sum  right  prompt")
    for item in prompts:
        for mode, build in MODES.items():
            result = ask(item["prompt"], build())
            total = result["input"] + result["tool_use"]
            totals[mode] += total
            right = "yes" if item["expect"] in result["called"] else "NO"
            print(
                f"{mode:<9} {result['input']:>6} {result['tool_use']:>9} {total:>7}  "
                f"{right:<5}  {item['prompt'][:40]}"
            )
    for mode, total in totals.items():
        dollars = total / 1_000_000 * PRICE_PER_MILLION_INPUT * 1000
        print(f"{mode}: {total} tokens, about ${dollars:.2f} per 1,000 runs of this set")


if __name__ == "__main__":
    main()

Reference

Which fields configure deferral?

FieldApplies toWhat it does
executiontool_search"server" (default) or "client"
defer_loadingfunction, mcp_serverWithholds the schema until a search loads it
short_descriptiondeferred toolsReplaces the description in the deferred listing

short_description is optional and valid only with defer_loading: true. A request takes one tool_search entry. Google's best practices say to start short_description with the service name and to prefix similar tool names by service, which is why the catalog here uses db_, k8s_ and billing_.

Which models can I run this on?

gemini-4-carbon, gemini-4-neon, gemini-4-hydrogen, gemini-4-argon, gemini-3.8-flash and gemini-3.6-flash. If you run Argon, Gemini 4 Argon pricing explains what the model costs before you add search turns on top.

What error do I get if I misconfigure it?

Error messageFix
tool_search tool must be specified when using defer_loading.Add {"type": "tool_search"} to tools
deferred loading is not supported for this modelSwitch to a supported model

What can't I do with tool retrieval?

Server and client retrieval can't be combined in one request. Tools Gemini has loaded stay in context and can't be unloaded. A turn can run more than one search.

What does deferral cost?

Tool retrieval has no separate charge. Deferred tools add their name and short_description (or the first sentence of description) to total_input_tokens. Implicit caching continues to work, but deferral can push a prompt below the minimum cacheable size. Pricing counts every token the model processes, including thinking tokens.

Common mistakes

  • Leaving out {"type": "tool_search"}, which makes a deferred tool error (Quickstart).
  • Sending tools only on the first request (Quickstart).
  • Deferring a tool your system instructions name (Quickstart).
  • Dropping parameters from a deferred tool. The catalog builder in step 2 sends the full schema for every tool, and deferral only withholds it until a search.
  • Calling generateContent. Tool retrieval works only on the Interactions API.
  • Judging by the input column alone. Read sum and right in the table (steps 6 and 7).

Next steps

  • Decide per tool, from your own numbers. If the deferred sum in measure.py beats the static sum on prompts like your real traffic, defer the long tail. Re-run measure.py whenever you add tools or edit the system prompt. For the same agent loop on another provider, with approval and cost limits, read the Claude tool use tutorial: build an agent loop with approval and cost limits.
  • Try client-side retrieval with execution: "client" on the tool_search entry, the alternative to the server-side mode used here.
  • Defer a remote MCP server's tools with defer_loading on an mcp_server entry. How to build an MCP server gives you one to point it at.
  • Check your caching: compare the dollar line with caching on and off.
  • Before you hand this agent's keys to a team, AI in the engineering workflow: tools, permissions and policy covers who gets access and under what rules.

FAQ

Does Gemini tool retrieval cost extra?

There's no separate charge. Deferred tools add only a name and description to total_input_tokens, but each server-side search makes an extra model call, reported in total_tool_use_tokens and billed at the input rate. With small libraries or contexts that search every turn, static loading can cost less.

Which Gemini models support tool retrieval?

gemini-4-carbon, gemini-4-neon, gemini-4-hydrogen, gemini-4-argon, gemini-3.8-flash and gemini-3.6-flash. Other models return deferred loading is not supported for this model.

Does tool retrieval work with generateContent?

No. It's available only on the Interactions API, and generateContent doesn't support it. It's also in preview and needs the v1beta API version.

Can I unload a tool after Gemini loads it?

No. Loaded tools stay in the context for the rest of the conversation. If a long session keeps loading schemas, measure whether starting a fresh interaction is cheaper.

Should I use server-side or client-side retrieval?

Start server-side, the default and the mode used here. Google recommends it for latency-sensitive apps, because searches run inside one API call. execution: "client" is the alternative, and you can't combine the two in one request.

Sources

Share this article

Author Image

HighCircl Editorial Team

The HighCircl editorial team writes about hiring software engineers, nearshore development, and engineering team building. Our articles draw on direct experience sourcing and placing senior developers across Poland, Hungary, Slovakia, Serbia, Slovenia, Romania, and Spain — and on candid conversations with the CTOs and engineering leads who hire them.

HighCircl is a nearshore engineering network that delivers matched candidate shortlists in 72 hours. Every piece of content we publish is informed by real engagement data: actual developer rates, real hiring timelines, and what separates engineering teams that scale cleanly from those that stall.