Before You Approve AI Automation, Demand an Evidence Gate

The immediate problem is not selecting an AI model. It is deciding whether an automation proposal has enough evidence to deserve implementation work at all.

That distinction matters when a team is being asked to approve an AI feature but the supporting material does not identify a model version, pricing, context limits, API behavior, privacy terms, or evaluation results. Without those details, architecture diagrams and delivery estimates rest on unknowns. The responsible decision is not to reject AI permanently. It is to keep the implementation decision open while the missing evidence is gathered.

For the source packet supplied for this article, no AI model or provider documentation is present. One source is a Go release-notes page; the other is a Rust funding page. Neither supplied item supports claims about AI capability, model economics, context limits, or provider behavior.

Verified fact: the available evidence cannot support a model-specific AI recommendation.

Recommendation: approve a bounded evidence-gathering exercise, not a production AI commitment. Compare an AI candidate with ordinary software before choosing an architecture.

The decision is AI automation versus a deterministic baseline

Start with a specific task rather than a general desire to “use AI.” A decision-ready problem statement should name:

• the input entering the workflow;
• the output the system must produce;
• the person or process using that output;
• the cost or consequence of an incorrect result;
• the cases that require human judgment;
• the information the system is allowed to process;
• the current non-AI way of completing the task.

This framing creates at least two implementation candidates.

The first is ordinary software: explicit validation, rules, state transitions, search, templates, queues, or a conventional user interface. The second is an AI-assisted path whose exact model and behavior still need to be verified.

Ordinary software should remain the default comparator when inputs are structured, desired outputs are deterministic, and exceptions can be expressed as maintainable rules. AI may deserve investigation when the useful input is difficult to reduce to stable rules, but that is only a reason to evaluate it. It is not evidence that it will satisfy the product, privacy, reliability, or cost constraints.

Interpretation: the correct first approval is often permission to compare approaches, not permission to ship one of them.

Missing model evidence is a delivery risk, not a documentation detail

A model-dependent design needs a named version and a supplied primary source for each material constraint. At minimum, the evidence record should cover:

• the exact model or API version under consideration;
• capability relevant to the stated task;
• input and output limits;
• documented API request and response behavior;
• pricing units and applicable conditions;
• data retention or processing terms relevant to the workflow;
• failure responses, rate constraints, and other operational behavior;
• the date on which each source was checked.

The current source packet contains none of that model-specific information. Consequently, this article cannot responsibly choose a provider, estimate token cost, describe a context window, or claim that a model can perform a particular task.

That absence should change the status of the proposal. It should not be hidden behind placeholders that look like confirmed design parameters.

Use four labels in the decision record:

• Verified fact: directly supported by an identified primary source.
• Calculation: derived from stated inputs with the formula visible.
• Assumption: temporarily accepted so an option can be explored.
• Interpretation: a judgment connecting the evidence to the decision.

For example, “the provider charges a particular amount” would require current provider evidence and a version or pricing reference. “Monthly cost equals completed tasks multiplied by average attempts and cost per attempt” is a calculation structure. It still needs verified inputs before it becomes an estimate.

Build the technical path around replaceable boundaries

While evidence is being collected, the technical design can remain provider-neutral without pretending that all providers behave alike.

A bounded path can contain these stages:

1. Input acceptance: validate type, size, required fields, authorization, and consent before any model call.
2. Data minimization: remove information that is not necessary for the task.
3. Deterministic baseline: attempt the rule-based or conventional workflow where it is sufficient.
4. AI candidate adapter: isolate provider-specific requests, responses, timeouts, and error handling behind a narrow interface.
5. Output validation: check structure and required constraints without assuming that plausible output is correct.
6. Human-review queue: route defined categories of output to an accountable reviewer.
7. Decision log: retain only the operational information permitted by the privacy policy and needed for evaluation.
8. Fallback: specify what happens when the AI path is unavailable, invalid, or outside its approved boundary.

This is a proposed architecture, not a description of a supplied provider. Its purpose is to preserve options. A provider adapter can be replaced; a privacy leak or workflow dependency is harder to reverse.

The deterministic path is not merely a fallback. It is the baseline against which the AI path must justify its additional variability, review load, privacy exposure, and cost.

Evaluate completed work, not attractive output

A useful evaluation begins before implementation. Write the acceptance criteria first, using examples representative of the actual task. The supplied metadata contains no task dataset, model results, or benchmarks, so no success threshold can be asserted here.

A practical evaluation plan should answer five questions.

What is the unit of evaluation?

Choose a unit tied to the workflow: one request classified, one draft prepared for review, or one record transformed. Avoid evaluating isolated fragments if the business needs a completed task.

What counts as acceptable?

Define observable conditions. These might include required fields, correct routing, policy compliance, or approval by a designated reviewer. The conditions should be selected by the team responsible for the workflow.

What is the baseline?

Measure the same units and acceptance conditions for the existing process or a deterministic implementation. Without that comparison, the evaluation only shows that the AI candidate produced outputs.

Who reviews ambiguous cases?

Reviewers need an explicit rubric and authority. If two reviewers can reasonably disagree, the evaluation should record that ambiguity rather than force certainty into the score.

What would stop the pilot?

Predefine invalidation conditions. Examples include unacceptable privacy exposure, inability to validate outputs, excessive review burden, unstable operational behavior, or unit cost outside the permitted range. These are categories for the team to quantify, not claims that any particular model exhibits them.

Do not select only convenient examples after seeing the outputs. Establish the evaluation set and exclusions before comparing candidates. When edge cases are especially consequential, include them deliberately and report them separately rather than blending them into an aggregate result.

Put the human-review boundary in the product design

“Human in the loop” is incomplete unless the loop has a trigger, owner, interface, and response time.

Define which outputs can proceed automatically and which must stop for review. The boundary should become stricter as the consequence of error increases. It may also depend on whether the output can be validated deterministically.

For every review route, document:

• what caused the review;
• what evidence the reviewer sees;
• what actions the reviewer may take;
• whether the original input can be edited or resubmitted;
• how corrections affect later evaluation;
• what happens when no reviewer is available.

Human review also has a cost. If every output requires substantial inspection, the AI step may shift work rather than remove it. That can still be useful, but the proposal should describe the value accurately.

Assumption to test: reviewers can identify unacceptable outputs consistently and with an acceptable amount of effort.

Invalidation condition: if reviewers cannot apply the rubric consistently, neither automation nor the evaluation result is ready for broader use.

Treat privacy as an input constraint

Privacy review should begin before prompts or integrations are designed. First inventory the fields entering the workflow. Then classify which fields are necessary, optional, prohibited, or eligible for transformation before external processing.

The decision record should identify:

• where input originates;
• whether a person expects it to be processed this way;
• which fields leave the existing system boundary;
• what is retained and for how long;
• who can inspect requests and outputs;
• how deletion and access requests would be handled;
• which claims still require provider documentation.

No provider privacy behavior can be stated from the supplied sources. Any production approval therefore remains conditional on obtaining and reviewing the relevant primary documentation.

A useful architectural rule is to minimize data before deciding how to send it. If the task can be completed with a smaller representation, that smaller representation should become a candidate input for evaluation.

Calculate unit economics without inventing a price

The absence of supplied pricing does not prevent the team from defining the calculation. It only prevents filling in unsupported numbers.

For an AI path, a basic cost model can be expressed as:

cost per accepted task = (model-call cost + infrastructure cost + review cost + retry cost) / accepted tasks

Expand the model where needed:

model-call cost = input usage cost + output usage cost + any documented fixed charges

review cost = reviewer time per task × loaded time cost

retry cost = failed or repeated attempts × cost per attempt

The inputs must be labeled. Provider pricing requires a current primary source. Usage requires measurement from the candidate workflow. Review time requires observation or an explicitly marked assumption. Accepted-task count must use the evaluation rubric, not raw API responses.

Compare that result with the deterministic baseline and current process using the same definition of an accepted task. A lower call price does not establish lower workflow cost if retries, review, or exceptions increase.

Invalidation condition: reject the estimate if a material input is unknown, sourced from a different version, or calculated using a different task definition from the baseline.

A practical evidence gate for approval

Before approving implementation, ask the proposal owner to provide:

• a narrow problem statement and current workflow;
• a deterministic baseline design;
• the exact model and API version, if AI remains a candidate;
• primary sources for capability, pricing, limits, API behavior, and privacy terms;
• a representative evaluation set;
• acceptance and rejection criteria written before testing;
• a human-review rubric and accountable owner;
• fallback and failure behavior;
• unit-cost formulas with each input labeled;
• explicit assumptions and a plan to verify them;
• conditions that would stop or reverse the recommendation.

If those items are unavailable, the decision should remain “not enough evidence.” That is not the same as “never.” It prevents an exploratory idea from being mistaken for a production commitment.

The current recommendation

Based only on the supplied metadata, approving a model-specific AI implementation would be unsupported. There is no verified model version, capability evidence, price, context limit, API behavior, privacy documentation, or evaluation result.

The defensible next step is a bounded discovery phase: define the task, build or document the deterministic baseline, obtain primary model documentation, prepare the evaluation rubric, and calculate costs only after the required inputs are available.

AI earns a place in the architecture when it clears that evidence gate under the team’s explicit criteria. Until then, ordinary software is not a lesser option. It is the necessary comparator.

Official sources

The supplied source set contains a Go documentation release-notes page and a Rust documentation funding page. These pages do not provide the model-specific evidence required for the AI automation decision described above.

If you need a neutral review of an automation proposal, I can help turn its assumptions into a technical audit, evaluation plan, and delivery decision without forcing AI into the solution.