Safety, Ethics & Compliance: NCP-AAI Domain 9 (5%)

Why agents raise the stakes

A chatbot that produces a bad answer has produced a bad answer. An agent that produces a bad answer may also have sent an email, issued a refund, or written to a database. Autonomy converts a content problem into an action problem, which is why safety in an agentic system is enforced around the tools as well as around the text.

Guardrails come in layers

Effective guardrails sit at more than one point. Input rails screen what reaches the model. Dialogue rails keep a conversation inside its intended scope. Tool rails decide which actions are permitted, with what arguments, and which need approval. Output rails check what is about to be returned. Each catches something the others cannot, and a design that names only one is usually the wrong answer.

Input rail    → screen incoming content before the model sees it
Dialogue rail → keep the conversation within allowed topics
Tool rail     → permit/deny actions, validate arguments, gate writes
Output rail   → check the response before it reaches the user

Prompt injection is an architectural problem

Injection happens when untrusted content — a retrieved document, a web page, a tool result — carries instructions the model then follows. It cannot be solved by prompting alone, because the model has no reliable way to distinguish instructions from data in its context. The durable defences are architectural: treat all retrieved and tool-returned content as untrusted data, keep tool permissions least-privilege, and require confirmation for consequential actions.

Data handling and PII

Agents accumulate context, and that context is often personal data. The controls are ordinary data-protection practice applied to a new surface: collect the minimum needed, redact or tokenise sensitive fields before they reach the model or the logs, be explicit about retention, and know which jurisdiction the processing happens in. Traces are a common leak — a system that redacts PII from prompts but logs the raw payload has not redacted anything.

Bias, transparency and auditability

Where an agent influences decisions about people, you need to be able to show what it did and why. That means a retained trace of the reasoning and actions behind a decision, evaluation that checks for disparate outcomes across groups rather than only average quality, and disclosure that the user is interacting with an AI system. Regulatory regimes differ in detail but converge on the same requirements: documentation, human oversight and traceability.

Exam tip

For any safety question, ask where the damage actually occurs. If the harmful step is an action, the correct control is a tool permission or an approval gate — not an output filter, which runs after the action has already happened.