What did the proof of concept prove?
A proof of concept tests a hypothesis under limited conditions. Finding correct specifications in a prepared set of product documents shows that the combination can answer those questions with those sources. The ability to handle all sales requests or to communicate directly with buyers requires separate evidence.
The NIST Generative AI Profile advises against generalizing model performance from narrow, unsystematic, or anecdotal evaluations. Use deployment-like conditions and verify the limitations of the use case and the sources supporting generated claims.[1]
Consider a hypothetical B2B distributor. It connects an assistant to product manuals, and the demo retrieves specifications and standard shipping instructions quickly. Support staff still open the original system and write replies by hand.
Live requests mix buyer-specific contract prices, updated stock, partial shipments, and possible cancellations. The manual holds none of this, and a submitted order number could refer to a different account. A written draft saves little when the operator has to recheck every fact from scratch.
Before changing the model, identify the missing business conditions.
| What the demo showed | What the live workflow needs | Evidence |
|---|---|---|
| Finds a specification | Retrieves the current version this account may see | Document version, permission and source passage |
| Writes a fluent reply | Operator can spot missing conditions and unsupported promises | Review checks, edits and elapsed time |
| Recognizes an order number | Order belongs to the authenticated customer | Identity and authorization results |
| Generates a reply quickly | Total lead time falls including review and rework | Record from intake to completion |
| Handles routine questions | Defers and routes missing or conflicting information | Exception and handoff records |
An empty evidence column identifies work the pilot still needs to do. Record the gap so the next investment can target it.
Which workflow should go first?
Here, AX means using AI and related technologies to change how work is performed and who is responsible for each step. A suitable first workflow is small enough to describe in one sentence: who completes what using which input.
The People + AI Guidebook recommends finding a real user need where AI adds value and considering augmentation as well as automation. It asks teams to weigh the consequences of mistakes alongside performance benefits.[2]
Use the table to compare candidate jobs. Restrict or defer a candidate where data use is impermissible or the cost of an error cannot be contained, regardless of volume.
| Criterion | Ask first | Favorable for a pilot | Narrow or defer |
|---|---|---|---|
| Frequency | How often does the same work recur? | Enough recorded examples to compare | Too few cases to evaluate |
| Delay | Where does staff time or waiting accumulate? | Clear search, classification, or drafting bottleneck | Bottleneck is external approval or physical delivery |
| Error cost | Who would be affected by a mistake? | Reviewable before release or easy to reverse | Immediate effects on money, rights, or safety |
| Verifiability | What determines a good outcome? | Source, business rule and completed state can be checked | Staff disagree on the correct answer or permissible range |
| Data and access | May the team use the required facts? | Sources, owners, effective dates and access are known | A shared account or unrestricted access would be required |
| Fallback | Who takes over unresolved work? | Existing workflow and an owner are available | Nobody owns failures or recovery |
The hypothetical distributor could start with drafts supported by public product documentation. Staff keep responsibility for setting a buyer's price or changing an order. If the account or order is unclear, start by confirming identity.
Give each step a suitable mechanism. Rules can apply an approved exchange rate or check required fields. AI can interpret a request phrased in ordinary language or find explanations in several documents. An authoritative system can validate permissions and compute the final amount.
Anthropic recommends starting with simple patterns. A predefined workflow gives predictable handling for bounded steps. More autonomous agents can expand the reachable tasks while adding cost and latency.[3]
Evidence connected from requirement to approval — Link requirements, tests, defect actions and approval records so teams judge quality consistently.
What counts as complete before anyone writes a prompt?
“Answer inquiries accurately” can mean different things to engineering and operations. The system may log success as soon as text appears. The operator may need the customer to receive a verified response and know what to do next.
Write a one-page pilot contract. The fields below are an example for operating teams.
- Input: authenticated buyer's inquiry, confirmed product identifier, and documents the buyer may see.
- Output: draft, source and location, unconfirmed conditions, and any follow-up questions.
- Complete: an operator confirms the evidence and customer, the approved reply is sent, and the case contains a record of delivery.
- Out of scope: setting new contract prices, approving refunds, changing quantities, and retrieving another account's data.
- Hold: source conflict, stale or unavailable state, unidentified product, or failed authorization.
- Owners: business rule owner, operator, reviewer, and the person empowered to decide exceptions.
A concrete definition exposes missing screens and records. Reviewers need source passages alongside the draft. A bad answer needs a traceable document version and run. If completed inquiries reenter the queue, check state handling outside the model.
NIST AI RMF connects governance, context mapping, measurement, and management. The proposed one-page contract applies that perspective to a workflow and should be judged by how it works in practice.[4]
Which facts are ready for use?
For every important fact, identify its source of truth, period of validity, and audiences authorized to see it.
A product manual may own specifications. Inventory owns current stock and a contract system owns buyer-specific prices. Copying all three into a document may simplify a search while obscuring update times and ownership. Map each fact to its owner and time of verification.
“Can it arrive this week?” requires at least a product, quantity, destination, recent stock check, and shipping cutoff. The draft can ask for missing facts or explain general policy. A firm arrival promise requires confirmation.
Agree how to resolve conflicts between a manual and a notice. An approved source state or escalation to the owner provides better evidence than the newest filename. Test whether restricted content stays out of retrieval, drafts, and summaries under actual permissions.
Reserve examples for final evaluation. Tuning repeatedly against familiar cases can mask poor generalization. Include ordinary requests and rare high-cost exceptions. Record the mix so a test set with deliberately frequent exceptions does not get presented as the live rate.
How do we compare total effort?
Establish the baseline before introducing AI. Record volume, active staff time, waiting and rework under the same definitions. A quiet week and a busy week can differ even when the process is identical.
Measure three layers.
| Layer | Question | Example measures |
|---|---|---|
| Output quality | Did the draft identify the right facts and unknowns? | Evidence match, omissions, unsupported promises, reviewer disagreement |
| Task execution | Did the work actually get done? | Completions among eligible requests, editing time, rework and handoffs |
| Operations | Was the feature available and recoverable? | Latency, dependency failure, cost, denials and manual recovery time |
Fix the denominators. Selecting 200 easy inquiries from a thousand means their completion rate describes that subset. Record total intake, eligible cases, actual use, approved completions, holds and failures. Keep escalations in the accounting.
Where possible, compare similarly difficult cases in the old and assisted flows. Without randomized assignment, record differences in request type, staff experience and timing. Track familiarization separately and include training in the cost.
Google Cloud's operations guide discusses monitoring outputs and real user feedback after deployment, including changes in the input distribution. A successful pilot remains evidence for the conditions the team measured.[5]
Hypothetical calculation: what is 120 hours of capacity worth?
These are illustrative estimates. The example assumes equal inquiry difficulty and the same previous average handling time.
| Item | Assumption or calculation | Monthly time |
|---|---|---|
| Existing workflow | 2,000 × 12 minutes | 400 hours |
| AI-assisted 60% | 1,200 × 5 minutes including review | 100 hours |
| Existing workflow for the remaining 40% | 800 × 12 minutes | 160 hours |
| Additional operations | Documentation, evaluation and exception management | 20 hours |
| Total after the change | 100 + 160 + 20 | 280 hours |
| Capacity freed | 400 − 280 | 120 hours |
At an internal rate of KRW 30,000 per hour, that capacity has an estimated value of KRW 3.6 million per month. With a further KRW 700,000 in monthly model, tool, and infrastructure costs, estimated net value is KRW 2.9 million. A one-time build, training, and evaluation cost of KRW 8.7 million gives a simple three-month payback under these assumptions.
The estimate depends on how that capacity is used. If salaries, outsourcing and overtime stay at their prior levels, the cash outflow may rise by KRW 700,000. Measure useful results such as clearing a backlog, shorter wait times or incremental revenue separately from cash savings.
The five minutes must average all eligible assisted cases, including correction and manual recovery after failure. Keep per-case operating work from being counted a second time in the extra 20 hours. Inspect slow outliers as well as the average.
Can a human reviewer genuinely intervene?
The reviewer needs the source facts and current state on the same screen. They also need the authority to stop delivery and retain their edits. An approval button alone provides little evidence of supervision.
Microsoft's Responsible AI Standard v2 identifies owners for oversight, asks that the system let them understand, intervene and stop as needed, and calls for testing those capabilities under realistic conditions. This is a Microsoft standard.[6]
Test whether the reviewer can check figures, recipients and promises against the underlying source, then correct, reject, or hand off the draft. Assign priorities when the review queue grows and give another operator a way to take over when the primary reviewer is away.
Use more than login counts to judge adoption. See where people abandon a draft: re-searching for evidence, switching screens, lacking permission, or losing changes. Capture the reason in the workflow so a complaint can become a fix.
Where does an exception go?
When the assistant cannot answer, the inquiry still needs a path to completion. Google's graceful-failure guidance describes determining the cause and giving users an actionable next step.[7]
For a distributor, log where automation stopped and use the previous system without duplicating work.
| Condition | Next action | Owner | Before resuming |
|---|---|---|---|
| Product or order unidentified | Hold the draft and show missing fields | Case operator | Confirm the identifiers match the inquiry |
| Two documents conflict | Show both sources and defer an answer | Document owner and workflow owner | Record approved rule and effective date |
| Source system unavailable | Surface the failed lookup and hand off manually | Operations | Reconcile pending and completed work |
| Attempted unauthorized access | Deny access and investigate scope | Security and data owners | Verify remediation and retest boundary |
| Delivery outcome unknown | Check delivery history before attempting again | Messaging operator | Confirm supplier and workflow state |
Run a fallback drill during the pilot. Check that work in progress survives a pause, a person can take it over, and completed actions cannot be repeated. Add the failure case to regression coverage once corrected.
What evidence can a 30-day pilot produce?
This schedule assumes a small support-drafting workflow. Data integration, permissions and volume may require longer. The elapsed time is one input to planning.
| Days | Work | Evidence needed to proceed |
|---|---|---|
| 1–5 | Observe work, select one task, define completion and exclusions, collect baseline | Owner and comparison measures identified |
| 6–10 | Check data sources and access, build a minimal flow, define examples and holds | Required facts and checks are clear |
| 11–17 | Compare offline, test ordinary and exceptional requests, check reviewer workflow | Critical gaps fixed and restricted scope agreed |
| 18–24 | Pilot with approved users and tasks, log time, rework, exceptions and cost | Actual work records and fallback results |
| 25–30 | Explain failures, estimate operating cost, decide expand, narrow or hold | Owner, stop conditions and next review date recorded |
Use these dispositions with thresholds tied to the baseline and business risk.
- Consider expansion: total effort improves within scope, access and evidence controls work, and staff can take over exceptions.
- Narrow: retain the subset with verified benefit and route difficult or costly groups back to the old flow.
- Hold: evidence, required data, or exception ownership is missing. Restore the preconditions and measure again.
- Stop and investigate: unauthorized disclosure, unauthorized external action, or duplicate orders trigger a predefined stop.
Rare events may be absent from a small pilot. Record exposure limits, detection, authority to stop, and the next review date.
What should the next meeting decide?
After the proof of concept, decide which operating conditions the team will commit to testing.
- Which task will change first, and who decides it is complete?
- Who owns the required data sources and access policies?
- What metric includes review, correction, rework and exceptions?
- What triggers a stop, and who takes over the manual workflow?
- Who approves an expansion and when is the next review?
Bring the current workflow and proof-of-concept results when scoping the pilot so the team can decide which data, integration, and QA checks come first.
IXC's Qoretix describes retaining test results, failure evidence, uncovered scope and retest records. Map those records to an observable user journey. Define the business criteria for AI responses, evaluation data and acceptable risk for each workflow.[8]
