Quillon Travel is adding suggested replies for human chat agents. A suggestion must appear within about one second while the agent is still reading the customer's message. Volume is about four million suggestions a month, and finance has fixed the budget per suggestion from day one. Each suggestion is a short, routine reply of one or two sentences based on a policy snippet supplied in the request. The team has an evaluation set of 500 past chats with approved replies.
Which starting approach should the architect recommend?
Show the answer and explanation
Correct answer
A. Start with the fastest, lowest-cost tier, test it on the evaluation set, and upgrade only where it falls short.
Each option
A Tight latency, high volume, a fixed unit budget and a straightforward, grounded task are exactly the cases the documentation lists for starting efficiency-first; the evaluation set confirms quality before any upgrade.
B A capability-first start (the page begins with Opus 5.5 and moves to Fable 5.1 only if evals fall short) is a valid strategy for complex reasoning or when accuracy outweighs cost; here the replies are routine and the budget is fixed from day one, so starting at the top tier conflicts with both cost and latency. "Most workloads start with Claude Opus 5.5" describes the general default; the same page lists tight latency, cost sensitivity and high-volume straightforward tasks as the cases for efficiency-first, and all hold here.
C fails first on cost: fast mode (available only on Opus-class models) is priced at a premium, which the fixed unit budget rules out. It also targets output speed rather than time to first token, which matters less for a one-to-two-sentence reply.
D The mid tier is not justified by any fact in the scenario, and judging quality from feedback after launch skips the evaluation set the team already has.
"For many applications, starting with a faster, more cost-effective model like Claude Haiku 5.5 can be the optimal approach:" with steps "Test your use case thoroughly." / "Upgrade only if necessary for specific capability gaps."
Efficiency-first is best for "Applications with tight latency requirements", "Cost-sensitive implementations", "High-volume, straightforward tasks"
Model matrix: "The lowest latency and price" → "Real-time applications, high-volume intelligent processing, cost-sensitive deployments"
Capability-first is best for "Complex reasoning tasks" and "Applications where accuracy outweighs cost considerations"
"Most workloads start with Claude Opus 5.5."
"Create benchmark tests specific to your use case - having a good evaluation set is the most important step in the process." and "Test with your actual prompts and data."
"Fast mode delivers up to 2.5x higher output tokens per second from Claude Opus 5.5, Claude Opus 5, and Claude Opus 4.8 at premium pricing."
"Speed benefits are focused on output tokens per second (OTPS), not time to first token (TTFT)"
Sample 2 of 3 · Topic 3 · Integration
Ashby Property Management generated its maintenance agent's tools from its internal API, one per endpoint: 34 tools. Several overlap: four tools return tenant data in different ways, and two open work orders, of which only the newer one records urgency. Logs show the agent often pages through every tenant to find one person and sometimes opens work orders without urgency. Three weeks ago the team added a system-prompt section naming the preferred tenant and work-order tools; the error rate did not change. Goal: improve the agent's tool-selection accuracy and efficiency.
Does this solution meet the goal?
Statement 1
Rewrite that system-prompt section in more explicit, emphatic wording.
Statement 2
Replace the overlapping endpoint tools with a few task-level tools, for example one that finds a tenant by name or unit and one that opens a work order with urgency, and remove the older duplicate.
Statement 3
Split the assistant into two subagents, one holding the four tenant tools and one holding both work-order tools, coordinated by an orchestrator.
Show the answer and explanation
Correct answer
Statement 1: No
Statement 2: Yes
Statement 3: No
Each option
Statement 1 The scenario shows that naming the preferred tools in the prompt did not change the error rate; the overlapping tools and the token-heavy listing tool are all still offered, so the source of ambiguity remains.
Statement 2 Endpoint-wrapping created overlapping tools and ambiguous decision points; task-level tools and removal of the duplicate give a minimal, non-overlapping set, and a lookup tool avoids paging through all tenants.
Statement 3 Each subagent still holds the same overlapping tools (four tenant tools, two work-order tools), so the ambiguous choices move into the subagents, while an orchestrator adds tokens and complexity.
"One of the most common failure modes we see is bloated tool sets that cover too much functionality" and "lead to ambiguous decision points about which tool to use"
"If a human engineer can't definitively say which tool should be used in a given situation, an AI agent can't be expected to do better."
"curating a minimal viable set of tools for the agent"
"A common error we've observed is tools that merely wrap existing software functionality or API endpoints."
"Too many tools or overlapping tools can also distract agents from pursuing efficient strategies."
"you might choose to implement a search_contacts or message_contact tool instead of a list_contacts tool." and "Tools can consolidate functionality, handling potentially multiple discrete operations (or API calls) under the hood."
Norvik Software is rolling Claude Code out to 70 developers.
For each requirement, select where the platform team should configure it. Each location may be used once, more than once, or not at all.
You chose:
You chose:
You chose:
You chose:
Show the answer and explanation
Correct answer
Description 1: Managed settings deployed by the organization
Description 2: Shared project settings committed to the repository
Description 3: Project-local settings kept out of version control
Description 4: The project's CLAUDE.md committed to the repository
Each option
Description 1 Managed settings: the rule must hold in every repository and resist every developer. Managed settings sit above all other levels; a deny rule cannot be carved out by an allow rule or a ! exception from another source. A project file would cover one repository and can be changed by anyone who commits to it; CLAUDE.md is context, not enforcement.
Description 2 Shared project settings: team permission rules and hooks for one repository belong in the committed project file, which every clone receives.
Description 3 Project-local settings: a personal override for one repository that stays out of commits; user settings would change the model in every project on the machine.
Description 4 Project CLAUDE.md: this is knowledge Claude cannot infer from the code (layout, naming), needed in every session and shared through version control; a permission rule or hook could block a wrong path but would not teach the convention.
Scope table: Managed "Everyone your organization deploys it to"; Shared project "Everyone working in the folder that contains it. In a git repository, commit it so teammates get it"; Project local "You, in this one project only."; User "You, in every project on this machine"
"Nothing in your own settings files or --settings overrides a managed key"
"Commit .claude/settings.json so everyone who clones the repository gets the same permissions, hooks, and plugins."
"To change a setting for yourself in one project without changing it for your teammates, save it in .claude/settings.local.json inside the project."
"Permission rules are enforced by Claude Code, not by the model. Instructions in your prompt or CLAUDE.md shape what Claude tries to do, but they don’t change what Claude Code allows."
"An allow rule can’t carve an exception out of a deny rule."
"A Read(!.env) in project settings or in --disallowedTools doesn’t cancel a Read(./.env) deny from managed settings or any other settings file."
Project instructions: "Team-shared instructions for the project Project architecture, coding standards, common workflows Team members via source control"
"Claude treats them as context, not enforced configuration. To block an action regardless of what Claude decides, use a PreToolUse hook instead."
"CLAUDE.md is a special file that Claude reads at the start of every conversation. Include Bash commands, code style, and workflow rules. This gives Claude persistent context it can’t infer from code alone."
Want full 63-question practice exams?
Plans include 3 to 10 timed practice exams with every option explained and your score per topic. From 29 USD.