On the Gemini API's Interactions API, you set defer_loading: true on the tools you rarely use and add {"type": "tool_search"} to tools. Gemini then loads a schema only when it needs one. Keep your most used tools loaded. With fewer than 20 stable tools, Google's own mode table puts you on static loading, because searches cost tokens too.
You'll build an ops desk agent with 30 tools, run six fixed prompts with and without deferral, and print a table of tokens from your own key. Tool retrieval is in preview and needs the v1beta API version, so the surface can change before general availability.
What you'll build
A Python command-line agent with a 30-function tool library: 4 core tools that always load and 26 that are deferred. The tool names, schemas and stub implementations are fixtures written for this tutorial, not Google's. measure.py runs the same six prompts in two modes, static (every schema sent) and deferred, and prints input tokens, tool-use tokens, their sum, and whether the right tool was called. Plan on about 45 minutes.
The project layout:
ops-desk-agent/
├── .env # your key, git-ignored
├── .env.example
├── .gitignore
├── agent.py # one prompt through the Interactions API
├── measure.py # runs both modes and prints the table
├── prompts.json # six fixed prompts
└── tools_catalog.py # the 30 tools and their stub implementationsHere's the shape of the finished output. These numbers are placeholders to show the layout, not a result: your run prints its own, and the last two lines show how the dollar figure is worked out.
mode input tool_use sum right prompt
static 3000 0 3000 yes Is the payments service up?
deferred 900 0 900 yes Is the payments service up?
static 3000 0 3000 yes Roll back the last release of the checko
deferred 900 1100 2000 yes Roll back the last release of the checko
static: 18000 tokens, about $27.00 per 1,000 runs of this set
deferred: 9800 tokens, about $14.70 per 1,000 runs of this setWhat you need before you start
- Python 3 with
venvandpip. The Interactions API needsgoogle-genai2.3.0 or later, and the install lines pin it. - A macOS or Linux shell. On Windows, use WSL.
- A Gemini API key from the API keys page in Google AI Studio, stored in the
GEMINI_API_KEYenvironment variable. The pricing page lists a free tier forgemini-4-hydrogen, the model used here, so a free key is enough to follow along.
measure.py sends about two dozen requests per run, because each prompt makes two calls in each mode. Keep that in mind if your key has a low rate limit.
Quickstart
Already have a key? Install the SDK, export the key, and run this:
pip install -U "google-genai>=2.3.0"
export GEMINI_API_KEY="your-api-key-here"from google import genai
client = genai.Client(http_options={"api_version": "v1beta"})
tools = [
{"type": "tool_search"},
{
"type": "function",
"name": "get_weather",
"description": "Get the current weather for a city.",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string", "description": "The city name."}},
"required": ["city"],
},
"defer_loading": True,
},
{
"type": "function",
"name": "get_stock_price",
"description": "Get the latest price for a stock ticker.",
"parameters": {
"type": "object",
"properties": {"ticker": {"type": "string", "description": "The ticker."}},
"required": ["ticker"],
},
"defer_loading": True,
},
]
interaction = client.interactions.create(
model="gemini-4-hydrogen",
input="What's the weather in Paris?",
tools=tools,
)
for step in interaction.steps:
print(step.type, getattr(step, "name", ""))Three things to get right before you build on it:
- Put
{"type": "tool_search"}intoolson every request. Without it,defer_loadingreturns an error. - Send the full
toolsarray again on every request, including the follow-up that returns a tool result. - Don't defer a tool your system instructions name. Keep it loaded.
How to build the agent and measure the saving
1. Create the project and install everything
Make the folder, a virtual environment, one install line for every package the tutorial uses, your key file, and the prompt fixture. Everything later builds on this folder.
In a terminal, from wherever you keep projects:
mkdir ops-desk-agent && cd ops-desk-agent
python3 -m venv .venv && source .venv/bin/activate
pip install -U "google-genai>=2.3.0" python-dotenvCreate .env.example, a template that's safe to commit:
GEMINI_API_KEY=Copy it to .env and paste your real key after the equals sign:
cp .env.example .envCreate .gitignore:
.env
.venv/
__pycache__/Create prompts.json. Four prompts need a deferred tool and two need a core tool. expect is the function the model should call:
[
{"prompt": "Is the payments service up?", "expect": "get_service_status"},
{"prompt": "Who is on call for the platform team?", "expect": "get_oncall"},
{"prompt": "Roll back the last release of the checkout service.", "expect": "deploy_rollback_release"},
{"prompt": "Show me the plan for customer cus_1042.", "expect": "billing_lookup_customer_plan"},
{"prompt": "Purge the CDN cache for /static/app.js.", "expect": "cdn_purge_cache"},
{"prompt": "How many connections does the orders database have right now?", "expect": "db_get_connection_count"}
]The SDK reads GEMINI_API_KEY from the environment but doesn't read .env files, so each Python file here calls load_dotenv() from python-dotenv first.
Checkpoint, in the same terminal:
pip show google-genaiYou should see something like (the version will differ):
Name: google-genai
Version: <installed version>If your prompt doesn't start with (.venv), re-run the source .venv/bin/activate line.
2. Generate the 30-tool library
Create tools_catalog.py. It holds one compact spec per tool and builds the full function schemas from them, so you don't paste 30 hand-written schemas. The fourth value in each spec marks the 4 core tools that step 5 keeps loaded. This file is tutorial glue, not Google code.
import json
SPECS = [
("get_service_status", "Get the current status of a service.", {"service": "string"}, True),
("get_oncall", "Get who is on call for a team.", {"team": "string"}, True),
("create_ticket", "Create a ticket in the tracker.", {"title": "string"}, True),
("search_runbook", "Search the runbooks by keyword.", {"query": "string"}, True),
("db_list_backups", "List backups for a database.", {"database": "string"}, False),
("db_restore_backup", "Restore a database from a backup.", {"database": "string", "backup_id": "string"}, False),
("db_get_connection_count", "Get the open connection count for a database.", {"database": "string"}, False),
("db_run_vacuum", "Run a vacuum on a database.", {"database": "string"}, False),
("k8s_restart_worker_pool", "Restart a Kubernetes worker pool.", {"pool": "string"}, False),
("k8s_scale_deployment", "Scale a Kubernetes deployment.", {"deployment": "string", "replicas": "integer"}, False),
("k8s_get_pod_logs", "Get recent logs for a pod.", {"pod": "string"}, False),
("k8s_list_namespaces", "List the namespaces in a cluster.", {"cluster": "string"}, False),
("billing_lookup_customer_plan", "Look up the plan a customer is on.", {"customer_id": "string"}, False),
("billing_issue_refund", "Issue a refund for an invoice.", {"invoice_id": "string"}, False),
("billing_list_invoices", "List the invoices for a customer.", {"customer_id": "string"}, False),
("billing_apply_credit", "Apply an account credit to a customer.", {"customer_id": "string", "amount": "integer"}, False),
("iam_rotate_api_key", "Rotate the API key of a service.", {"service": "string"}, False),
("iam_list_users", "List the users on a team.", {"team": "string"}, False),
("iam_disable_user", "Disable a user account.", {"user": "string"}, False),
("iam_grant_role", "Grant a role to a user.", {"user": "string", "role": "string"}, False),
("dns_get_record", "Get the DNS record for a hostname.", {"hostname": "string"}, False),
("dns_update_record", "Update the DNS record for a hostname.", {"hostname": "string", "value": "string"}, False),
("cdn_purge_cache", "Purge a path from the CDN cache.", {"path": "string"}, False),
("cdn_get_hit_rate", "Get the CDN cache hit rate for a domain.", {"domain": "string"}, False),
("alerts_silence_alert", "Silence an alert for some minutes.", {"alert_id": "string", "minutes": "integer"}, False),
("alerts_list_firing", "List the alerts firing for a team.", {"team": "string"}, False),
("deploy_get_last_release", "Get the last release of a service.", {"service": "string"}, False),
("deploy_rollback_release", "Roll back the last release of a service.", {"service": "string"}, False),
("cost_get_monthly_spend", "Get the monthly cloud spend for an account.", {"account": "string"}, False),
("cost_list_idle_instances", "List idle instances in an account.", {"account": "string"}, False),
]
def _build(name, description, params, defer):
tool = {
"type": "function",
"name": name,
"description": description,
"parameters": {
"type": "object",
"properties": {
p: {"type": t, "description": "The " + p.replace("_", " ") + "."}
for p, t in params.items()
},
"required": list(params),
},
}
if defer:
tool["defer_loading"] = True
return tool
def static_tools():
return [_build(n, d, p, False) for n, d, p, _core in SPECS]
def run_tool(name, args):
return json.dumps({"tool": name, "args": dict(args or {}), "status": "ok"})
if __name__ == "__main__":
tools = static_tools()
print("tools:", len(tools))
print("schema characters:", len(json.dumps(tools)))static_tools() returns every tool with its full parameters schema. run_tool is a stub that echoes the call back as JSON, so the agent has something to execute.
Checkpoint, from ops-desk-agent/:
python tools_catalog.pyYou should see something like (your character count will differ):
tools: 30
schema characters: <number>That character count is the weight static loading puts on every request.
3. Send every schema on every request
Create agent.py. It sends one prompt with all 30 schemas attached, which is the baseline you'll measure deferral against. The client is pinned to v1beta because the Interactions API's tool retrieval needs it.
from dotenv import load_dotenv
from google import genai
from tools_catalog import static_tools
load_dotenv()
MODEL = "gemini-4-hydrogen"
client = genai.Client(http_options={"api_version": "v1beta"})
if __name__ == "__main__":
interaction = client.interactions.create(
model=MODEL,
input="Is the payments service up?",
tools=static_tools(),
)
for step in interaction.steps:
print(step.type, getattr(step, "name", ""))
print("total_input_tokens:", interaction.usage.total_input_tokens)Checkpoint:
python agent.pyYou should see something like (the token count will differ):
function_call get_service_status
total_input_tokens: <number>The model asked you to call get_service_status instead of answering. If you see an authentication error, check that .env sits in this folder and holds your real key.
4. Run the tool and return the result
Replace the whole of agent.py. Now ask() runs the loop: it executes each function_call, sends the result back with previous_interaction_id, and adds up the usage of every request. It checks the steps rather than status, because Google's guide says status isn't always requires_action.
import sys
from dotenv import load_dotenv
from google import genai
from tools_catalog import run_tool, static_tools
load_dotenv()
MODEL = "gemini-4-hydrogen"
MAX_TURNS = 5
client = genai.Client(http_options={"api_version": "v1beta"})
def ask(prompt, tools):
interaction = client.interactions.create(model=MODEL, input=prompt, tools=tools)
step_types = []
called = []
tokens_in = 0
tokens_tool = 0
for _ in range(MAX_TURNS):
usage = interaction.usage
tokens_in += usage.total_input_tokens or 0
tokens_tool += getattr(usage, "total_tool_use_tokens", None) or 0
step_types += [step.type for step in interaction.steps]
calls = [step for step in interaction.steps if step.type == "function_call"]
if not calls:
break
results = []
for call in calls:
called.append(call.name)
results.append(
{
"type": "function_result",
"name": call.name,
"call_id": call.id,
"result": [{"type": "text", "text": run_tool(call.name, call.arguments)}],
}
)
interaction = client.interactions.create(
model=MODEL,
input=results,
previous_interaction_id=interaction.id,
tools=tools,
)
return {
"text": interaction.output_text or "",
"steps": step_types,
"called": called,
"input": tokens_in,
"tool_use": tokens_tool,
}
if __name__ == "__main__":
result = ask(sys.argv[1], static_tools())
print(result["text"])tools=tools goes into the follow-up request too, as the Quickstart said. Both usage fields are optional, so each is added with or 0 and a request that ran no search still works.
Checkpoint:
python agent.py "Is the payments service up?"You should see something like (the wording will differ):
The payments service reports status ok.If the output is empty, print result["steps"] to see where the loop stopped.
5. Defer the long tail
Add deferral to tools_catalog.py, then let agent.py pick a mode. The 26 non-core tools get defer_loading: True, the 4 core tools stay loaded, and {"type": "tool_search"} goes in the array. Gemini then sees only each deferred tool's name and description until it asks for a schema.
Append this to the end of tools_catalog.py, below the if __name__ == "__main__": block:
def deferred_tools():
tools = [{"type": "tool_search"}]
tools += [_build(n, d, p, not core) for n, d, p, core in SPECS]
return toolsIn agent.py, replace the import line from tools_catalog import run_tool, static_tools with:
from tools_catalog import deferred_tools, run_tool, static_toolsThen replace the if __name__ == "__main__": block at the bottom of agent.py with:
if __name__ == "__main__":
tools = deferred_tools() if sys.argv[1] == "deferred" else static_tools()
result = ask(sys.argv[2], tools)
print("steps:", " > ".join(result["steps"]))
print("called:", result["called"])
print("text:", result["text"])
print("total_input_tokens:", result["input"], "total_tool_use_tokens:", result["tool_use"])Checkpoint:
python agent.py deferred "Roll back the last release of the checkout service."You should see something like (the wording and counts will differ):
steps: tool_search_call > tool_search_result > function_call > model_output
called: ['deploy_rollback_release']
text: The checkout service was rolled back.
total_input_tokens: <number> total_tool_use_tokens: <number>The first three step types are the ones to look for: a search, its result, then the real call. Searches run inside the same API call and previous_interaction_id keeps the history on Google's side, so you never handle or resend tool_search_call yourself.
6. Compare both modes on the same prompts
Create measure.py. It runs every prompt in prompts.json in both modes and prints one row per run, plus a total and a dollar estimate per mode.
Each server-side search triggers an extra model call with the full context plus the loaded schemas. Those tokens land in total_tool_use_tokens, billed at the input rate, so the sum is the figure that matters. With a small library, or a long context that searches every turn, static loading can win.
Hydrogen's paid input price is $1.50 per million tokens on the Standard tier, which is the constant in the file. Check Google's pricing page before you trust the dollar line.
import json
from agent import ask
from tools_catalog import deferred_tools, static_tools
PRICE_PER_MILLION_INPUT = 1.50
MODES = {"static": static_tools, "deferred": deferred_tools}
def main():
with open("prompts.json") as f:
prompts = json.load(f)
totals = {mode: 0 for mode in MODES}
print("mode input tool_use sum right prompt")
for item in prompts:
for mode, build in MODES.items():
result = ask(item["prompt"], build())
total = result["input"] + result["tool_use"]
totals[mode] += total
right = "yes" if item["expect"] in result["called"] else "NO"
print(
f"{mode:<9} {result['input']:>6} {result['tool_use']:>9} {total:>7} "
f"{right:<5} {item['prompt'][:40]}"
)
for mode, total in totals.items():
dollars = total / 1_000_000 * PRICE_PER_MILLION_INPUT * 1000
print(f"{mode}: {total} tokens, about ${dollars:.2f} per 1,000 runs of this set")
if __name__ == "__main__":
main()Checkpoint:
python measure.pyYou should see 12 rows and two total lines in the layout from What you'll build, with your own numbers.
7. Run it end to end and check the edge cases
Read the table before you trust it. Three things tell you whether deferral earned its place on your prompts: sum per row, right per row, and the totals.
First, look at the two core-tool prompts (the first two in prompts.json). Run one in deferred mode and confirm the steps hold no tool_search_call:
python agent.py deferred "Is the payments service up?"You should see something like:
steps: function_call > model_output
called: ['get_service_status']
text: The payments service reports status ok.
total_input_tokens: <number> total_tool_use_tokens: 0A core tool needs no search, so deferred mode should pay nothing for one here. Then run a prompt that matches no tool:
python agent.py deferred "Book a meeting room for Friday."You should see something like (the wording will differ):
steps: tool_search_call > tool_search_result > model_output
called: []
text: I don't have a tool for booking rooms.
total_input_tokens: <number> total_tool_use_tokens: <number>A search that matches no deferred tool gives you a text response, not a structured error, so handle called == [] as a normal outcome. Finally, scan the right column. Google publishes no accuracy figure for retrieval, so a NO in deferred mode where static said yes is the finding to chase, and your own prompts are the only place to find it.
Your files now match the ones in Full code.
How the pieces fit
your CLI (agent.py, measure.py)
| tools = core schemas + deferred names/descriptions + tool_search
| GEMINI_API_KEY (read from .env, never sent to a browser)
v
Interactions API (v1beta, gemini-4-hydrogen)
| tool_search_call > tool_search_result > function_call
| (searches resolve inside one API call)
v
your tool functions (run_tool in tools_catalog.py)
| function_result sent back with previous_interaction_id
v
final output_textClean up
Leave the virtual environment:
deactivateTo remove the project, delete the ops-desk-agent folder, which also deletes .env. If you made the API key only for this tutorial, revoke it in Google AI Studio.
Full code
Steps 1 to 6 build every file below. The project layout is in What you'll build.
Install and run, from ops-desk-agent/ with the virtual environment from step 1 active:
pip install -U "google-genai>=2.3.0" python-dotenv
cp .env.example .env # then put your real key in .env
python measure.py.env.example:
GEMINI_API_KEY=.gitignore:
.env
.venv/
__pycache__/prompts.json:
[
{"prompt": "Is the payments service up?", "expect": "get_service_status"},
{"prompt": "Who is on call for the platform team?", "expect": "get_oncall"},
{"prompt": "Roll back the last release of the checkout service.", "expect": "deploy_rollback_release"},
{"prompt": "Show me the plan for customer cus_1042.", "expect": "billing_lookup_customer_plan"},
{"prompt": "Purge the CDN cache for /static/app.js.", "expect": "cdn_purge_cache"},
{"prompt": "How many connections does the orders database have right now?", "expect": "db_get_connection_count"}
]tools_catalog.py:
import json
SPECS = [
("get_service_status", "Get the current status of a service.", {"service": "string"}, True),
("get_oncall", "Get who is on call for a team.", {"team": "string"}, True),
("create_ticket", "Create a ticket in the tracker.", {"title": "string"}, True),
("search_runbook", "Search the runbooks by keyword.", {"query": "string"}, True),
("db_list_backups", "List backups for a database.", {"database": "string"}, False),
("db_restore_backup", "Restore a database from a backup.", {"database": "string", "backup_id": "string"}, False),
("db_get_connection_count", "Get the open connection count for a database.", {"database": "string"}, False),
("db_run_vacuum", "Run a vacuum on a database.", {"database": "string"}, False),
("k8s_restart_worker_pool", "Restart a Kubernetes worker pool.", {"pool": "string"}, False),
("k8s_scale_deployment", "Scale a Kubernetes deployment.", {"deployment": "string", "replicas": "integer"}, False),
("k8s_get_pod_logs", "Get recent logs for a pod.", {"pod": "string"}, False),
("k8s_list_namespaces", "List the namespaces in a cluster.", {"cluster": "string"}, False),
("billing_lookup_customer_plan", "Look up the plan a customer is on.", {"customer_id": "string"}, False),
("billing_issue_refund", "Issue a refund for an invoice.", {"invoice_id": "string"}, False),
("billing_list_invoices", "List the invoices for a customer.", {"customer_id": "string"}, False),
("billing_apply_credit", "Apply an account credit to a customer.", {"customer_id": "string", "amount": "integer"}, False),
("iam_rotate_api_key", "Rotate the API key of a service.", {"service": "string"}, False),
("iam_list_users", "List the users on a team.", {"team": "string"}, False),
("iam_disable_user", "Disable a user account.", {"user": "string"}, False),
("iam_grant_role", "Grant a role to a user.", {"user": "string", "role": "string"}, False),
("dns_get_record", "Get the DNS record for a hostname.", {"hostname": "string"}, False),
("dns_update_record", "Update the DNS record for a hostname.", {"hostname": "string", "value": "string"}, False),
("cdn_purge_cache", "Purge a path from the CDN cache.", {"path": "string"}, False),
("cdn_get_hit_rate", "Get the CDN cache hit rate for a domain.", {"domain": "string"}, False),
("alerts_silence_alert", "Silence an alert for some minutes.", {"alert_id": "string", "minutes": "integer"}, False),
("alerts_list_firing", "List the alerts firing for a team.", {"team": "string"}, False),
("deploy_get_last_release", "Get the last release of a service.", {"service": "string"}, False),
("deploy_rollback_release", "Roll back the last release of a service.", {"service": "string"}, False),
("cost_get_monthly_spend", "Get the monthly cloud spend for an account.", {"account": "string"}, False),
("cost_list_idle_instances", "List idle instances in an account.", {"account": "string"}, False),
]
def _build(name, description, params, defer):
tool = {
"type": "function",
"name": name,
"description": description,
"parameters": {
"type": "object",
"properties": {
p: {"type": t, "description": "The " + p.replace("_", " ") + "."}
for p, t in params.items()
},
"required": list(params),
},
}
if defer:
tool["defer_loading"] = True
return tool
def static_tools():
return [_build(n, d, p, False) for n, d, p, _core in SPECS]
def run_tool(name, args):
return json.dumps({"tool": name, "args": dict(args or {}), "status": "ok"})
if __name__ == "__main__":
tools = static_tools()
print("tools:", len(tools))
print("schema characters:", len(json.dumps(tools)))
def deferred_tools():
tools = [{"type": "tool_search"}]
tools += [_build(n, d, p, not core) for n, d, p, core in SPECS]
return toolsagent.py:
import sys
from dotenv import load_dotenv
from google import genai
from tools_catalog import deferred_tools, run_tool, static_tools
load_dotenv()
MODEL = "gemini-4-hydrogen"
MAX_TURNS = 5
client = genai.Client(http_options={"api_version": "v1beta"})
def ask(prompt, tools):
interaction = client.interactions.create(model=MODEL, input=prompt, tools=tools)
step_types = []
called = []
tokens_in = 0
tokens_tool = 0
for _ in range(MAX_TURNS):
usage = interaction.usage
tokens_in += usage.total_input_tokens or 0
tokens_tool += getattr(usage, "total_tool_use_tokens", None) or 0
step_types += [step.type for step in interaction.steps]
calls = [step for step in interaction.steps if step.type == "function_call"]
if not calls:
break
results = []
for call in calls:
called.append(call.name)
results.append(
{
"type": "function_result",
"name": call.name,
"call_id": call.id,
"result": [{"type": "text", "text": run_tool(call.name, call.arguments)}],
}
)
interaction = client.interactions.create(
model=MODEL,
input=results,
previous_interaction_id=interaction.id,
tools=tools,
)
return {
"text": interaction.output_text or "",
"steps": step_types,
"called": called,
"input": tokens_in,
"tool_use": tokens_tool,
}
if __name__ == "__main__":
tools = deferred_tools() if sys.argv[1] == "deferred" else static_tools()
result = ask(sys.argv[2], tools)
print("steps:", " > ".join(result["steps"]))
print("called:", result["called"])
print("text:", result["text"])
print("total_input_tokens:", result["input"], "total_tool_use_tokens:", result["tool_use"])measure.py:
import json
from agent import ask
from tools_catalog import deferred_tools, static_tools
PRICE_PER_MILLION_INPUT = 1.50
MODES = {"static": static_tools, "deferred": deferred_tools}
def main():
with open("prompts.json") as f:
prompts = json.load(f)
totals = {mode: 0 for mode in MODES}
print("mode input tool_use sum right prompt")
for item in prompts:
for mode, build in MODES.items():
result = ask(item["prompt"], build())
total = result["input"] + result["tool_use"]
totals[mode] += total
right = "yes" if item["expect"] in result["called"] else "NO"
print(
f"{mode:<9} {result['input']:>6} {result['tool_use']:>9} {total:>7} "
f"{right:<5} {item['prompt'][:40]}"
)
for mode, total in totals.items():
dollars = total / 1_000_000 * PRICE_PER_MILLION_INPUT * 1000
print(f"{mode}: {total} tokens, about ${dollars:.2f} per 1,000 runs of this set")
if __name__ == "__main__":
main()Reference
Which fields configure deferral?
| Field | Applies to | What it does |
|---|---|---|
execution | tool_search | "server" (default) or "client" |
defer_loading | function, mcp_server | Withholds the schema until a search loads it |
short_description | deferred tools | Replaces the description in the deferred listing |
short_description is optional and valid only with defer_loading: true. A request takes one tool_search entry. Google's best practices say to start short_description with the service name and to prefix similar tool names by service, which is why the catalog here uses db_, k8s_ and billing_.
Which models can I run this on?
gemini-4-carbon, gemini-4-neon, gemini-4-hydrogen, gemini-4-argon, gemini-3.8-flash and gemini-3.6-flash. If you run Argon, Gemini 4 Argon pricing explains what the model costs before you add search turns on top.
What error do I get if I misconfigure it?
| Error message | Fix |
|---|---|
tool_search tool must be specified when using defer_loading. | Add {"type": "tool_search"} to tools |
deferred loading is not supported for this model | Switch to a supported model |
What can't I do with tool retrieval?
Server and client retrieval can't be combined in one request. Tools Gemini has loaded stay in context and can't be unloaded. A turn can run more than one search.
What does deferral cost?
Tool retrieval has no separate charge. Deferred tools add their name and short_description (or the first sentence of description) to total_input_tokens. Implicit caching continues to work, but deferral can push a prompt below the minimum cacheable size. Pricing counts every token the model processes, including thinking tokens.
Common mistakes
- Leaving out
{"type": "tool_search"}, which makes a deferred tool error (Quickstart). - Sending
toolsonly on the first request (Quickstart). - Deferring a tool your system instructions name (Quickstart).
- Dropping
parametersfrom a deferred tool. The catalog builder in step 2 sends the full schema for every tool, and deferral only withholds it until a search. - Calling
generateContent. Tool retrieval works only on the Interactions API. - Judging by the input column alone. Read
sumandrightin the table (steps 6 and 7).
Next steps
- Decide per tool, from your own numbers. If the deferred sum in
measure.pybeats the static sum on prompts like your real traffic, defer the long tail. Re-runmeasure.pywhenever you add tools or edit the system prompt. For the same agent loop on another provider, with approval and cost limits, read the Claude tool use tutorial: build an agent loop with approval and cost limits. - Try client-side retrieval with
execution: "client"on thetool_searchentry, the alternative to the server-side mode used here. - Defer a remote MCP server's tools with
defer_loadingon anmcp_serverentry. How to build an MCP server gives you one to point it at. - Check your caching: compare the dollar line with caching on and off.
- Before you hand this agent's keys to a team, AI in the engineering workflow: tools, permissions and policy covers who gets access and under what rules.
FAQ
Does Gemini tool retrieval cost extra?
There's no separate charge. Deferred tools add only a name and description to total_input_tokens, but each server-side search makes an extra model call, reported in total_tool_use_tokens and billed at the input rate. With small libraries or contexts that search every turn, static loading can cost less.
Which Gemini models support tool retrieval?
gemini-4-carbon, gemini-4-neon, gemini-4-hydrogen, gemini-4-argon, gemini-3.8-flash and gemini-3.6-flash. Other models return deferred loading is not supported for this model.
Does tool retrieval work with generateContent?
No. It's available only on the Interactions API, and generateContent doesn't support it. It's also in preview and needs the v1beta API version.
Can I unload a tool after Gemini loads it?
No. Loaded tools stay in the context for the rest of the conversation. If a long session keeps loading schemas, measure whether starting a fresh interaction is cheaper.
Should I use server-side or client-side retrieval?
Start server-side, the default and the mode used here. Google recommends it for latency-sensitive apps, because searches run inside one API call. execution: "client" is the alternative, and you can't combine the two in one request.
