CCAR-P sample question: an agent with too many overlapping tools

An original sample question by CERTain·exam, based only on Anthropic's public documentation. CERTain·exam is not affiliated with Anthropic.

Domain 3 · Integration (19% of CCAR-P, the largest domain in the exam guide effective until December 6, 2026; 21% from December 7, 2026). Objective covered: "Evaluate tool/agent configuration for capability bloat."

This is a Yes/No item. You judge each of three proposed fixes against one goal, and each statement is right or wrong on its own.

The question

Ashby Property Management generated 34 tools for its maintenance agent from its internal API, one per endpoint. Several overlap. Four tools return tenant data in different ways, and two tools open work orders, but only the newer one records urgency. Logs show the agent often pages through every tenant to find one person and sometimes opens work orders without urgency. Three weeks ago the team added a system-prompt section naming the preferred tenant and work-order tools, but the error rate did not change. Goal: improve the agent's tool-selection accuracy and efficiency.

Does this solution meet the goal? Answer Yes or No for each statement.

  1. Rewrite that system-prompt section in more explicit, emphatic wording.
  2. Replace the overlapping endpoint tools with a few task-level tools, for example one that finds a tenant by name or unit and one that opens a work order with urgency, and remove the older duplicate.
  3. Split the agent into two subagents, one holding the four tenant tools and one holding both work-order tools, coordinated by an orchestrator.
Show the answer and explanation

Answer

Statement 1: No · Statement 2: Yes · Statement 3: No

How to read this question

  1. Situation. The team generated 34 tools, one per API endpoint, and some overlap. The agent pages through all tenants and opens work orders without urgency. A prompt fix was already tried and did not change the error rate.
  2. What is asked. Whether each change, on its own, improves tool-selection accuracy and efficiency.
  3. Statement by statement.
    • 1 (No). The scenario shows that naming the preferred tools in the prompt did not change the error rate. The overlapping tools and the token-heavy listing tool are all still offered, so the source of ambiguity remains.
    • 2 (Yes). Wrapping endpoints created overlapping tools and ambiguous decision points. Task-level tools, with the duplicate removed, give a minimal, non-overlapping set, and a lookup tool avoids paging through all tenants.
    • 3 (No). Each subagent still holds the same overlapping tools (four tenant tools, two work-order tools), so the ambiguous choices move into the subagents, while an orchestrator adds tokens and complexity.
  4. Rule to remember. When an agent picks the wrong tool, fix the tool set before you touch the prompt or the architecture. Use fewer task-level tools with no overlap.

Sources (Anthropic, quotes checked Oct 23, 2026; page checked Oct 27, 2026)

  • Effective context engineering for AI agents:
    • "One of the most common failure modes we see is bloated tool sets that cover too much functionality" and "lead to ambiguous decision points about which tool to use".
    • "If a human engineer can't definitively say which tool should be used in a given situation, an AI agent can't be expected to do better."
    • "curating a minimal viable set of tools for the agent".
  • Writing tools for agents:
    • "A common error we've observed is tools that merely wrap existing software functionality or API endpoints."
    • "Too many tools or overlapping tools can also distract agents from pursuing efficient strategies."
    • "you might choose to implement a search_contacts or message_contact tool instead of a list_contacts tool." and "Tools can consolidate functionality, handling potentially multiple discrete operations (or API calls) under the hood."
  • Building effective agents:
    • "Poka-yoke your tools. Change the arguments so that it is harder to make mistakes."
    • "we recommend finding the simplest solution possible, and only increasing complexity when needed."
  • How we built our multi-agent research system:
    • "multi-agent systems use about 15× more tokens than chats."

More practice

Questions

Where does this question come from?

It's an original sample question that CERTain·exam wrote based only on Anthropic's public engineering articles and the public exam guide.

What does CCAR-P Domain 3 (Integration) cover?

According to the exam guide effective until December 6, 2026, Integration (19%) covers capability bloat in tool and agent configuration, authentication and authorization gaps, accuracy-latency trade-offs, observability at scale, RAG pipelines with chunking and indexing, retrieval strategies, choosing between MCP, API/CLI and agent-to-agent integration, and progressive discovery vs monolithic context. From December 7, 2026, the domain weighs 21%, and its objectives are rewritten. See what changes on December 7, 2026.

Get your free practice exam

First, sign in with your email.