6 worked examples, with the reasoning spelled out. These are written for this page rather than taken from the live bank — showing a real question would spend it as a measurement for anyone who read this first.
Implement tool use and environment interaction — 20-25%
A coding agent needs to read an internal documentation service while it works on issues. What is the right way to give it that access?
- APaste the documentation into every issue the agent is assigned
- BGive the agent the password of a developer who already has access
- CConfigure the service as an MCP server for the agent, with scoped credentialsCorrect
- DLet the agent search the public web and hope the documents are indexed there
WhyAn MCP server exposes the service to the agent as tools and resources through a standard protocol, and scoped credentials limit what it can reach. Pasting documents goes stale, borrowing a person's password breaks accountability and least privilege, and internal documents are not on the public web.
Prepare agent architecture and SDLC processes — 15-20%
An agent's pull requests are often rejected because they change more than the issue asked for. What should change first?
- ATighten the issue's scope and acceptance criteriaCorrect
- BGive the agent a larger context window so it understands the codebase better
- CAllow it to merge its own pull requests whenever its tests pass
- DAssign the same issue to two agents and keep whichever change is smaller
WhyScope creep is a task-definition problem before it is a model problem. A bounded issue with explicit acceptance criteria tells the agent where to stop and gives the reviewer a test to apply. Self-merging removes the review that caught the problem, and racing two agents doubles the work.
Manage memory, state, and execution — 10-15%
An agent resumes a task that spans several days and redoes work it had already finished. What was missing?
- AA model with a more recent training cut-off date
- BA faster runner for the CI workflow it uses
- CA higher temperature so that it varies its approach
- DA durable record of progress it could read backCorrect
WhyState that must survive between sessions has to live outside the model — a plan or checklist in the issue or pull request that the agent reads when it starts again. Training data, runner speed and sampling settings have nothing to do with what this task has already done.
Perform evaluation, error analysis, and tuning — 15-20%
Which evidence best shows whether an agent's change did what the issue asked?
- AThe agent's own summary saying that the task is complete and ready to merge
- BThe acceptance criteria checked against the change, plus passing required checksCorrect
- CThe number of files the agent touched while it was making the change
- DHow quickly the agent opened its pull request after being assigned the issue
WhyEvaluation has to tie the output back to the intent: the acceptance criteria say what was wanted, and required checks say the change does not break anything. The agent's own summary is a claim rather than evidence, and file counts and speed say nothing about correctness.
Orchestrate multi-agent coordination — 15-20%
Three agents work in parallel on one feature and their pull requests keep conflicting. Which practice would have prevented most of it?
- AGiving all three agents write access to the same branch from the start
- BRunning the agents one after another without any shared plan
- CSplitting the work into non-overlapping subtasks before they startedCorrect
- DChoosing a larger model for each of the three agents
WhyConflicts in parallel work are mostly a planning failure: two agents were given overlapping ground. Decomposing the work into subtasks that do not touch the same code, each owned by one agent, removes most collisions before they happen. A shared branch makes them worse.
Implement guardrails and accountability — 10-15%
Which action should an agent never be able to complete without explicit human approval?
- AOpening a draft pull request with a proposed fix
- BAdding a comment that summarises its proposed plan on the issue it was assigned
- CRunning the repository's test suite on its own branch
- DChanging the branch protection rules on the default branchCorrect
WhyBranch protection is one of the controls that keeps agent output in check, so an agent able to change it unsupervised could remove its own guardrails. Drafting a pull request, commenting and running tests are low-risk and reviewable, and should not need anyone's approval.