8 worked examples, with the reasoning spelled out. These are written for this page rather than taken from the live bank — showing a real question would spend it as a measurement for anyone who read this first.
Agent Architecture and Design — 15%
An agent must complete a multi-step research task that may take several minutes and involve dozens of tool calls. Which architecture choice most reduces the risk of it looping indefinitely?
- AIncrease the context window so the whole history of the run stays in scope for every step
- BA step budget and an explicit termination condition checked by the orchestratorCorrect
- CA lower temperature, so the agent makes fewer unusual choices between its steps
- DRetry every failed tool call up to five times before letting the agent move on
WhyLooping is a control-flow problem, so the fix belongs in the control flow rather than in the model. A budget the orchestrator enforces, plus a condition that defines "done", bounds the work regardless of what the model decides next. A bigger window just lets a loop run longer, and aggressive retries make it worse.
Evaluation and Tuning — 13%
You need to evaluate an agent that answers open-ended questions from internal documents. Exact-match scoring against reference answers is producing scores that do not match human judgement. What is the appropriate next step?
- ARewrite the reference answers to be much shorter so that exact match succeeds more often
- BSwitch the headline metric to response latency, because latency is objective and easy to measure
- CScore with a rubric-based judge on groundedness and completeness, calibrated against human labelsCorrect
- DRemove the evaluation set entirely and rely on thumbs-up feedback from users in production
WhyExact match punishes correct answers that are worded differently, so it disagrees with humans by design. A rubric judge scores the properties you actually care about — is it grounded in the source, does it answer the whole question — and calibrating it against human labels is what stops you trusting a judge nobody checked.
Deployment and Scaling — 13%
An agentic service behind a single model endpoint starts timing out under load. Traces show most latency is spent waiting on sequential tool calls, not on generation. What addresses the cause?
- AParallelise independent tool calls and cache repeated lookupsCorrect
- BMove to a larger model deployment with a higher token throughput limit
- CIncrease the request timeout so that fewer requests fail under peak load
- DReduce the max output tokens so that each generation step finishes sooner
WhyThe traces name the bottleneck, and it is not generation — so changing the model or trimming output tokens addresses something that was never slow. Independent tool calls can run concurrently and repeated lookups need not run twice. Raising the timeout hides the symptom while making every queued request worse.
Safety, Ethics, and Compliance — 5%
An agent with access to a payments API is being deployed to handle customer refund requests. Which control belongs in the design rather than in the prompt?
- AAn instruction in the system prompt telling the agent to refuse any refund above a set threshold
- BA system-prompt reminder to be careful with customer money and to double-check every amount
- CA post-hoc daily report listing every refund the agent issued, reviewed by the finance team
- DA hard limit enforced outside the model, with human approval above itCorrect
WhyAnything in the prompt is guidance the model usually follows and can be talked out of, so it is not a control. A limit enforced in the calling code holds regardless of what the agent was persuaded to attempt, and a daily report is detection after the money has gone.
Agent Development — 15%
An agent calls an internal tool that expects structured arguments. A few percent of calls fail because the model emits arguments that do not match the tool's schema. What is the most robust fix?
- AAdd a line to the system prompt telling the model to always produce valid arguments
- BCatch the exception and return an empty result so the run never stops
- CValidate arguments against the tool schema and return the error to the model to retryCorrect
- DChange the tool to accept free-text input and parse its arguments with regular expressions
WhyA prompt instruction lowers the failure rate but cannot guarantee anything. Validating against the schema catches every bad call deterministically, and handing the validation error back gives the model what it needs to correct itself. Swallowing the error hides a real failure, and free text makes parsing harder, not easier.
Cognition, Planning, and Memory — 10%
A support agent handles long conversations. Once the history outgrows the context window, it starts forgetting account details the customer gave at the start. Which approach fits best?
- APersist key facts to structured memory and retrieve them each turnCorrect
- BRestart the conversation from scratch whenever the context window fills up
- CAsk the customer to repeat their account details every few turns of the chat
- DSend the full transcript every turn and let the provider truncate the oldest messages
WhyThe facts that matter are few and stable, so they belong in memory the agent controls rather than in a transcript that gets truncated from the front — which is exactly where the account details sit. Restarting or asking again pushes the system's limitation onto the customer.
Knowledge Integration and Data Handling — 10%
A retrieval-augmented agent answers questions from a policy library, but it often cites superseded versions of a policy that are still in the index. What addresses the root cause?
- AIncrease top-k so that more candidate documents reach the model on every question
- BSwitch to a larger embedding model so that similarity scores become more precise
- CAdd an instruction asking the model to prefer whichever document sounds the most recent
- DAttach version and effective-date metadata at ingestion and filter on it at retrievalCorrect
WhyOld and new versions of a policy are semantically almost identical, so no amount of similarity tuning separates them, and more top-k retrieves more of both. Version is a fact about the document, so it belongs in metadata captured at ingestion and enforced as a filter — not left to the model to guess.
NVIDIA Platform Implementation — 7%
A team needs programmable input and output rails for an LLM application on NVIDIA's stack — for example, refusing off-topic requests and checking responses before they reach the user. Which component is built for this?
- ANVIDIA NIM inference microservices
- BNeMo GuardrailsCorrect
- CTensorRT-LLM
- DTriton Inference Server
WhyNeMo Guardrails exists to add programmable rails around an LLM application — topical, safety and dialogue rails on what goes in and comes out. NIM packages and serves models, TensorRT-LLM optimises inference, and Triton serves models at scale; all three sit underneath the application rather than governing its conversations.