Which workflow should come first?

Start with a task whose inputs and outputs you can describe clearly. Controlled-document search or drafting a security-questionnaire response can be a useful candidate. Begin with approved internal material and a reviewable output. Clinical decisions and autonomous changes to production systems require a different level of evaluation and governance.

Ask the people doing the work where they lose time. Then measure a baseline: how long the task takes, what makes an answer acceptable, and how often the available documents fail to answer the question. A polished demonstration is not a baseline.

Define the access boundary

An assistant should only retrieve material that its user is permitted to access. Document the identity model, source permissions, vendor data handling, and retention requirements. Decide what the system must do when a source is unavailable, outdated, or outside the user's access.

  • Name an accountable workflow owner.
  • Specify approved sources and excluded data.
  • Define permitted tools and actions.
  • Keep consequential actions behind explicit human approval.
  • Record where inputs, outputs, and logs may be stored.

Test failure, not just fluency

Build a small evaluation set from representative questions, with expected evidence and acceptable answers. Include questions the sources cannot answer, conflicting versions, and requests to cross access boundaries. Measure correctness, source support, review effort, and task completion time.

NIST's Generative AI Profile is a useful reference for identifying and managing risks across the lifecycle. Adapt that reference to the actual workflow and people involved. A framework is a starting point for decisions, rather than proof that a system is safe.

Make the review visible

A draft should clearly identify its sources and uncertainties. Assign a reviewer who can accept, correct, or reject it. For a questionnaire workflow, the reviewer should check that the answer describes current practice and that sharing the cited material is permitted.

Agree on a stop or expand decision

Before implementation, decide what would justify continuing. That might mean meeting an agreed accuracy threshold while reducing review time and preserving access controls. If the pilot does not meet its criteria, document the limitation and change the scope before expanding it.