Agentic Search Boundary Review Checklist

Use this before you let an Elastic-backed agent touch a workflow that matters.

The point is not to approve a clever demo. The point is to confirm that the retrieval path, tool boundary, and refusal behavior are strong enough to survive production review.

1. Retrieval boundary

• The workflow starts with retrieval, not action.
• The system can show which documents or snippets shaped the answer.
• The retrieved set is intentionally narrow rather than a giant context dump.
• Stale or lower-quality documents do not outrank the source of truth.
• Retrieval results can be explained after the fact with citations or trace output.

2. Tool registry

• The tool list is explicit in code or configuration.
• There is at least one read-only path that answers without mutation.
• Any write-capable tool is tightly scoped to one bounded action.
• The agent cannot discover new tools at runtime.
• The tool contract makes clear what the tool can return and what it cannot do.

3. Refusal and escalation behavior

• The system distinguishes between retrieval-only, search-plus-tool, and workflow-required requests.
• The system can ask for clarification instead of guessing.
• The system can refuse or escalate requests that imply live mutation.
• Draft creation is sandboxed and does not silently trigger external side effects.
• Workflow-required requests do not bypass the boundary just because the model sounds confident.

4. Access control and leakage checks

• Role-based filtering is applied before the final answer is assembled.
• A stronger restricted match cannot leak through a weaker visible fallback.
• Metadata lookups do not expose ownership or sensitivity for hidden documents.
• Restricted results leave an audit trail in notes or trace data without exposing the content itself.
• At least one test exists for contractor or lower-privilege access.

5. Trace and debugging

• Every request has a request ID.
• The trace shows the chosen route.
• The trace shows top retrieval hits.
• The trace shows tool name and tool outcome.
• The trace shows the final state such as answered, drafted, blocked, or needs clarification.
• Latency and failure notes are visible somewhere an operator can inspect them.

6. Definition of done

You are in decent shape when all of these are true:

• A retrieval-only request returns a grounded answer with citations.
• A search-plus-tool request uses only an allow-listed tool.
• A workflow-style request is blocked, escalated, or sandboxed instead of executed live.
• A lower-privilege user does not get a misleading fallback answer for a stronger restricted match.
• A reviewer can read one trace and explain what happened.

Signoff

Before release or internal rollout, record:

Boundary-reviewed, traceable, and safe enough for the next stage of testing.

• Owner:
• Second reviewer:
• Date: