Agent Tool Calling Best Practices: Why Your Agent Picks the Wrong Tool

Agent Tool Calling Best Practices: Why Your Agent Picks the Wrong Tool

Agent tool calling best practices are what stand between a working demo and an agent that quietly calls the wrong tool in production. Anthropic's engineering team measured a single rewritten description cutting one tool result from 206 tokens to 72, and Claude Code now hard-caps every tool response at 25,000 tokens because the bleed is real. Your agent fails four ways, wrong tool, wrong arguments, runaway loops, token bleed, and each one has a fix you can ship this week.

Key takeaways:

• Agent tool calling fails in exactly four ways: wrong tool, wrong arguments, runaway loops, and token bleed.
• Tool descriptions are the only instruction the model sees at selection time, so they fix most wrong-tool calls.
• Flat, task-shaped schemas with validated inputs eliminate most wrong-argument failures.
• Concise tool responses and an eval loop on every change keep token cost and regressions measurable.

Why does agent tool calling fail in production?

Agent tool calling fails in four ways: the model picks the wrong tool, writes wrong arguments, spins in a runaway loop, or bleeds tokens through fat responses. Each failure hits a different step of the call loop, so fix order matters. Start with selection, because a wrong tool choice poisons every step after it.

The full prescription at a glance:

Work through them in this order. Practices 1 and 2 take an afternoon and remove most of the wrong-tool and wrong-argument failures you see today. A tool description is not documentation. It is the only instruction the model gets at selection time.

Phase 1: Design tools the model can actually use

The cheapest reliability gains in agent tool calling sit in your tool definitions, not your prompts or your model choice. The model never reads your API docs or your README. It sees a name, a description string, and a JSON schema, and it decides from those alone. Get those three right and selection accuracy moves before you touch anything else.

Practice 1: Write descriptions the model can act on

Write tool descriptions as instructions to the model, not as API documentation. A description that satisfies a human developer ("REST wrapper for the users endpoint") gives the model nothing to decide on. Anthropic's engineering guide on writing tools and their tool definition best practices both push the same pattern: say when to use the tool, what it returns, and when NOT to use it.

{
  "name": "get_user",
  "description": "Fetches a user profile. Use ONLY when you already have a user_id. Do NOT use to search or list users; call search_users instead. Returns name, email, plan. Errors if user_id is not a valid UUID."
}

versus the version most teams ship:

{
  "name": "get_user",
  "description": "Gets a user."
}

Two rules do most of the work here. First, name parameters so their meaning is unambiguous: user_id, never user or id, because user invites the model to pass a name or an email where a UUID belongs. Second, state exclusions explicitly. "Do NOT use to search users" prevents more wrong-tool calls than any amount of positive description, because models confuse overlapping tools far more often than they misunderstand single, clearly bounded ones. For the provider-level mechanics of how these definitions reach OpenAI, Anthropic, and Google APIs, see our multi-provider function calling guide.

Practice 2: Keep schemas flat and task-shaped

Keep input schemas flat, with every field the task actually needs and none it does not. Nested objects with optional branches are where wrong-argument failures breed: the model must infer structure it never sees examples of. The OpenAI function calling guide accepts arbitrary JSON Schema, but permissive is not the same as reliable.

{
  "name": "create_ticket",
  "parameters": {
    "type": "object",
    "properties": {
      "ticket": {
        "type": "object",
        "properties": {
          "details": {
            "type": "object",
            "properties": {
              "title": { "type": "string" },
              "meta": { "type": "object" }
            }
          }
        }
      }
    }
  }
}

Flatten it to the task:

{
  "name": "create_ticket",
  "parameters": {
    "type": "object",
    "properties": {
      "title": { "type": "string" },
      "priority": { "type": "string", "enum": ["low", "medium", "high"] },
      "assignee_id": { "type": "string" }
    },
    "required": ["title", "priority"]
  }
}

Enums beat free text for any field with a bounded set of values. Required arrays beat optional everything. If the model almost always needs a field, make it required in the tool schema even when your API calls it optional. You are not mirroring your API. You are designing a surface one specific model can fill correctly.

Phase 2: Manage the tool set, not just the tools

Individual tool quality stops being enough once an agent carries more than a handful of tools, because selection errors grow with the size of the list the model reads.

Practice 3: Wrap multi-step API sequences into atomic tools

Collapse any fixed sequence of API calls into one atomic tool. Anthropic's engineering post uses schedule_event and get_customer_context as the model: one call that does the whole job beats three calls the agent must chain correctly every time. Each link in a chain is another turn where the model can stall, retry wrongly, or loop.

# What the agent does WITHOUT an atomic tool: 3 calls, 3 chances to fail
calendar = call_tool("list_calendars", {})
free = call_tool("find_free_slot", {"calendar_id": calendar["items"][0]["id"], "duration": 30})
call_tool("create_event", {"calendar_id": calendar["items"][0]["id"], "start": free["start"]})

# One atomic tool: the sequence lives in your code, not the model's head
call_tool("schedule_event", {"duration": 30, "attendees": ["[email protected]"]})

The rule of thumb: if the model must always call B after A, A and B are one tool wearing two costumes.

Practice 4: Namespace, prune, and filter tools dynamically

Namespace every tool name and show each agent only the subset its current task needs. Generic names collide the moment you connect two integrations. Picture an agent wired to two MCP servers that both expose a tool called search: two identical verbs, no way to tell them apart. Anthropic documents measurable eval gains from prefix namespacing:

Pruning matters as much as naming. A support agent does not need its billing tools loaded while it answers a password question. The planner-worker pattern, where a planner routes a task to a worker that loads only the relevant tools, is the standard fix; LangGraph's dynamic tool loading how-to walks the implementation. How many tools is too many? Treat 5-10 per agent as a working range, not a law: accuracy degrades as the list grows, and the cure is filtering, not a bigger model. If you are choosing the routing and filtering layer itself, compare your options in our roundup of the best function calling libraries.

Phase 3: Control what comes back and what goes out

The loop runs both directions, and most teams only engineer the outbound half. What your tools return determines how much of the context window survives to the next turn, and what your validation rejects determines whether the model learns from its mistakes or repeats them.

Practice 5: Return concise, high-signal results

Return the smallest result the model can act on, with human-readable identifiers instead of raw IDs. Anthropic's engineering post documents a tool whose default result ran 206 tokens; a concise response_format setting cut the same result to 72 tokens, roughly a third of the size. Multiply that by dozens of calls per task and it decides whether your agent finishes at all.

// Before: 206 tokens (shape per Anthropic's documented example)
{
  "status": "success",
  "data": {
    "id": "8f14e45f-ceea-3f9c-a2f3-90c1b5e0a7d2",
    "object": "task", "created_at": "2026-07-02T09:14:00Z",
    "updated_at": "2026-07-11T16:40:12Z", "completed_at": null,
    "assignee": {"id": "c9a1...f2", "object": "user"},
    "projects": [{"id": "b7d3...91", "object": "project"}],
    "permalink": "https://app.asana.com/0/.../f"
  }
}

// After: 72 tokens
{ "task": "Fix login redirect", "assignee": "Dana Kim", "project": "Web App", "due": "2026-07-20" }

Two more details from the same source: Anthropic supports a response_format enum (detailed versus concise) on tool definitions, so you can declare the shape you want rather than parsing the firehose. And Claude Code caps tool responses at 25,000 tokens, a hard ceiling that truncates bloated results either way. Anthropic also reports, as their finding, that resolving UUIDs to semantic names measurably reduced retrieval hallucinations, which is why the "after" payload above says "Dana Kim" and not c9a1...f2. Fat responses are a cost problem too; see our guide to reduce LLM API costs for the full picture.

Practice 6: Validate every call and make errors teach the model

Validate every tool call server-side and return errors that contain the fix. Martin Fowler's piece on function calling frames it bluntly: never trust the model's output. It will pass strings where enums belong and invent IDs that do not exist.

def create_ticket(args):
    if args.get("priority") not in {"low", "medium", "high"}:
        return {"error": f"priority must be one of: low, medium, high. Got '{args.get('priority')}'. Pass priority='medium' for normal issues."}
    if not is_valid_uuid(args.get("assignee_id")):
        return {"error": "assignee_id must be a UUID. Call list_team_members to get valid IDs, then retry."}
    return db.create_ticket(**args)

The error string is the whole game. Compare:

# Unhelpful: the model retries the same bad call
{"error": "invalid input"}

# Helpful: the model knows exactly what to change
{"error": "priority must be one of: low, medium, high. Got 'urgent'. Use 'high'."}

Every validation error your tool returns is a prompt you are writing for the model's next attempt. Errors that name the constraint and point to the corrective tool turn a retry loop into a one-shot recovery. This is also your first security line of defense; our LLM guardrails guide covers it in depth.

Phase 4: How do you make it safe, then make it measurable?

Safety and measurement are the same phase because an ungated destructive action and an unmeasured regression both surface as incidents you could not see coming. Gate the actions that cannot be undone, then instrument everything so the next tool change is a decision with evidence behind it, not a hope.

Practice 7: Gate destructive actions behind a human

Separate read tools from write tools and put a human confirmation gate on anything destructive. The MCP specification's tool annotations exist for exactly this: destructiveHint marks tools that perform destructive updates, and openWorldHint flags tools that touch external systems, so clients can prompt before executing. Use them.

The failure mode is not hypothetical. Laurent Kubaski documented a case, in his July 2025 tool-calling write-up with the original report linked, where a user asked Copilot in Excel to act on row 4 and the agent acted on row 8 instead. No confirmation gate stood between the wrong row and the write. The fix is the pattern AWS documents for Bedrock Agents: the agent prepares the action, returns it for approval, and executes only after a human confirms. Cursor does the same for file edits. Scope credentials to read-only where reading is all the task needs, and treat confirmation gates as part of your injection surface, the topic of our prompt injection prevention guide.

Practice 8: Run an evaluation loop on every tool change

Run a small evaluation suite before and after every tool change, and read the metrics in a fixed order. Paragon's optimization guide proposes a four-metric frame worth adopting:

Anthropic's tool evaluation cookbook, built on real Slack and Asana MCP evals, shows what good and bad eval tasks look like:

# Weak: vague, many valid paths, impossible to score
"Use the Asana tools to organize some work."

# Strong: one correct tool, checkable arguments, binary outcome
"Create a task titled 'Renew TLS cert' in project 'Infra' assigned to [email protected], due 2026-08-15. Expect exactly one create_task call with those four fields."

Our interpretation, labeled as such: the published numbers give you the order to work in. check tool correctness first, because Anthropic's own measurements show description and naming changes move it directly (the 206-to-72 token rewrite, the UUID-to-name hallucination finding), and leave task efficiency for last, since it mostly reflects failures the first three metrics already caught. For the starter suite, design 15-30 tasks, two or three per tool, each with a single expected call and a binary pass condition. That size is enough to catch a regression from a description rewrite without a week of labeling, and we read the cookbook's Slack and Asana setup as evidence that a suite this small is the intended starting point, not a shortcut. The deeper mechanics live in our guide to evaluating AI agents in production, and if your eval results say the tools themselves are fine but the orchestration is not, that is when to revisit your framework choice against the best AI agent frameworks.

Agent tool calling vs MCP: what's the difference?

MCP is a transport and registry standard, not a reliability layer, so the same eight practices apply whether your tools arrive over MCP or are defined inline. Native tool calling is the model-provider contract: how the model emits a tool_call and reads a tool_result. MCP standardizes how tools reach the model; it does nothing about whether the model picks the right one.

An MCP server that exposes a tool named search with the description "searches things" fails identically to an inline function defined the same way. Fix the definition, then worry about the transport. Our Model Context Protocol guide covers the protocol side end to end.

How Techsy applies these eight practices

On every client agent build, we enforce three of these before anything else ships: descriptions written as instructions (Practice 1), validation gates on every write tool (Practice 6), and an eval suite that runs before deploy, not after an incident (Practice 8). Those three cover wrong-tool calls, wrong-argument calls, and the regressions that reintroduce both, which is where every production agent incident we have debugged started. The other five practices follow as the agent grows. If your agent is past the demo stage and picking the wrong tools, get a free consultation and we will tell you which of the eight to fix first.

Frequently Asked Questions

What is agent tool calling?

Agent tool calling is the mechanism where an LLM decides to invoke an external function, emits a structured tool_call, and waits for your code to return a tool_result it can reason over. It is what turns a chat model into an agent that can query databases, call APIs, and take actions: the model chooses the tool and arguments, your executor runs them.

How does the agent tool calling loop work?

The loop has five steps: the user request reaches the model, the model selects a tool and writes a tool_call, your executor runs it, a tool_result returns to the model, and the model either answers or issues another call. That cycle repeats until the task is done. The four failure modes in this guide each live at a specific step of this loop.

Why does my agent pick the wrong tool?

Usually because two tools overlap and their descriptions do not say which is which. The model selects from names and descriptions alone, so "gets a user" versus "finds users" reads as interchangeable. Fix it with exclusion lines ("do NOT use to search"), namespaced names, and fewer tools in context. Kubaski's four-model test showed even strong models misroute on ambiguous lists.

How do I force a tool calling agent to structure its output?

Constrain the schema, not the prompt. Use enums for bounded fields, required arrays for anything the task needs, and flat objects over nested ones. For the final answer rather than the tool call, provider features like OpenAI's structured outputs and Anthropic's tool-choice modes force a specific shape. Our structured outputs guide covers both paths with code.

Agent tool calling vs MCP: what's the difference?

Native tool calling is the contract between your code and one model provider: the tool_call and tool_result message format. MCP is a protocol layer that standardizes how tools are discovered and delivered to any compatible client. MCP changes the plumbing, not the reliability. A badly described tool fails the same way over either path, as our Model Context Protocol guide explains.

How many tools is too many for an LLM agent?

Treat 5-10 tools per agent as a working range, not a law. Selection accuracy degrades as the visible list grows, especially when names or descriptions overlap. The fix is not a bigger model but filtering: load only the subset the current task needs, using a planner-worker split. Namespace everything so two integrations never both expose a bare search.

What is the best model for tool calling?

There is no single answer, and published benchmarks age badly in this space. Frontier models from OpenAI, Anthropic, and Google all clear basic tool-use tasks, while smaller models paired with well-designed tools often complete tasks nearly as often at a fraction of the token cost. Build the 15-30 task eval suite from Practice 8 and test candidates against your own tools.

How do I reduce token cost from tool calling?

Cut what comes back. Return concise, high-signal results instead of raw API payloads: Anthropic documented a 206-to-72 token cut from one response_format change. Resolve UUIDs to names, drop fields the model never uses, and remember every tool result re-enters the context window on every following turn. Fewer calls, via atomic tools, removes whole results from the bill.

How do I evaluate tool calling quality?

Score four metrics in order: tool correctness (right tool?), input accuracy (valid arguments?), task completion (goal achieved?), and task efficiency (token and call count?). Write 15-30 tasks, each expecting one specific call with checkable arguments and a binary pass condition. Run the suite before and after every tool change so a description rewrite never ships unmeasured.

Conclusion

Diagnose before you optimize. Your agent picks the wrong tool for one of four reasons, and three of the eight practices above, descriptions, flat schemas, and filtering, fix the selection failures that drive most production incidents. Start there, because they cost an afternoon and they are why this problem is fixable at all. Keep validation errors informative, gate anything destructive behind a human, and run the eval loop on every change so you measure before you swap models. The wrong-tool problem is not a model problem. It is a tool-design problem, and you own the design.