Approve AI Features Only After the Evidence Gate

A product team has identified a workflow that might benefit from AI. The pressure is to choose a model, connect an API, and demonstrate progress. But there is a more important decision to make first: does the available evidence justify using AI at all?

For the material supplied with this article, the answer is not yet.

The sources identify an official Python release announcement and an Inside Java contributor page. They do not document an AI model, model version, price, context limit, API contract, evaluation result, privacy control, or operating cost. That absence is not a minor research gap. It prevents a responsible model recommendation.

The practical response is an AI feature approval gate: a short, explicit review that compares the proposed AI path with ordinary software and stops implementation when essential evidence is missing.

What the supplied evidence establishes—and what it does not

The supplied metadata identifies Python Insider as the official blog of the Python core development team. Its linked announcement says that Python 3.12.14, 3.11.16, and 3.10.21 are available (Python Insider).

The second source is an Inside Java page for Nicolai Parlog. The supplied metadata describes Inside Java as publishing news and views from members of Oracle’s Java team (Inside Java).

Those are the verified boundaries of the provided material. Neither source, as represented by the supplied metadata, supports claims about:

• an AI model’s capabilities;
• a particular model version;
• token or request pricing;
• context limits;
• structured output or tool-calling behavior;
• latency, availability, or rate limits;
• retrieval-augmented generation performance;
• agent behavior;
• privacy, retention, or data residency;
• benchmark or production results.

This distinction matters because a programming-language release and a contributor page may be relevant to a future implementation stack, but they are not evidence that an AI architecture is appropriate. Technology availability does not establish product fit.

Verified fact: The supplied sources concern Python releases and an Inside Java contributor page.

Interpretation: They may inform general implementation research, but they cannot support AI model selection or an estimate of AI operating economics.

Assumption: The proposed feature has not yet been approved and can still be implemented with either deterministic software or an AI component.

If that assumption is false—for example, if an organization has already committed to a specific model under documented constraints—the evidence gate still applies, but it becomes an architecture review rather than an initial approval review.

Start with a failure that users can recognize

“Add AI” is not a testable problem statement. An approval request should begin with a specific failure in the current workflow.

A useful statement has four parts:

1. Actor: Who encounters the problem?
2. Task: What are they trying to complete?
3. Failure: What prevents reliable completion today?
4. Acceptance condition: What observable result would count as adequate?

For example, a hypothetical internal request might say:

Support operators need to locate the relevant internal procedure for an incoming case. The current navigation does not consistently lead them to the applicable document. A proposed system must return a traceable source or abstain, and an operator must approve the result before it affects a customer.

This is an example structure, not a reported business case or measured outcome. It creates a decision that can be evaluated. It also leaves room for ordinary software.

Before considering retrieval-augmented generation, an agent, or any model-backed workflow, compare simpler options:

• improved information architecture;
• metadata filters;
• deterministic search;
• a guided form;
• rule-based routing;
• a conventional workflow with explicit states;
• a curated index of approved procedures.

Ordinary software is the better choice when explicit rules can meet the acceptance condition, inputs are sufficiently structured, and predictable behavior matters more than flexible interpretation. This is a decision rule, not a claim that deterministic systems always cost less or perform better.

The recommendation changes when a documented evaluation shows that the simpler options cannot meet the defined requirement and a specific AI approach can do so within the accepted review, privacy, and cost boundaries.

The evidence record required before model selection

If the problem survives the ordinary-software comparison, the team can open a model evidence record. Every material capability claim should be tied to an exact version and a supplied primary source.

For this decision, the minimum record is:

The current source set cannot fill any model-specific row. The correct entries are “not supplied,” not estimates inferred from memory or another model version.

This is especially important for products marketed through rolling aliases. If a primary source documents one version, its statements should not be silently applied to a different version. The same discipline applies to pricing units and context limits: a calculation is only useful when its inputs refer to the component that will actually run.

Evaluate the workflow, not an isolated response

A polished example output is not an evaluation method. The test needs to represent the feature’s actual decision boundary.

For the hypothetical procedure-finding workflow, an evaluation plan could define categories without inventing performance targets:

• requests with one clearly applicable procedure;
• requests with several plausible procedures;
• requests containing incomplete information;
• requests for which no approved procedure exists;
• documents the requester is not permitted to access;
• retrieved material that conflicts or has unclear status.

The team should decide in advance what is being scored. Possible dimensions include source traceability, abstention behavior, policy compliance, and whether the result is appropriate for human review. The exact thresholds must come from the product’s risk tolerance and operational requirements; none are supplied here.

Keep four kinds of statements separate in the evaluation report:

• Verified facts: Versioned behavior documented by a primary source or directly observed under a declared test procedure.
• Calculations: Values derived from stated inputs and formulas.
• Interpretations: Engineering or product judgments based on those facts and calculations.
• Assumptions: Inputs that remain unverified and may change the result.

That separation makes disagreement productive. A reviewer can challenge an assumption without disputing a documented fact, or question an interpretation without rewriting the calculation.

Put human review at a named boundary

“Human in the loop” is too vague for approval. The design must identify who reviews what, at which point, and with what authority.

For the hypothetical support workflow, the boundary might be: the system may suggest a procedure and expose its source, but an authorized operator must confirm applicability before any customer-facing action. If no suitable source is found, the workflow must escalate rather than fabricate a substitute.

That boundary is an assumption until the responsible product, security, legal, and operational stakeholders approve it. It also has cost implications. Review is not free merely because it is performed by an existing employee.

A practical review design should answer:

• Which outputs may be used without approval?
• Which outputs always require approval?
• What evidence does the reviewer see?
• Can the reviewer reject or correct the output?
• Where are corrections recorded?
• What happens when no reviewer is available?
• Which cases require escalation outside the normal workflow?

If these questions have no owner, the feature is not ready for implementation approval.

Calculate unit economics without inventing inputs

No provider price, token volume, request rate, or review duration is present in the supplied sources. A numeric estimate would therefore be fabricated. The useful deliverable at this stage is the calculation structure.

For each completed workflow:

estimated unit cost = model input cost + model output cost + retrieval cost + infrastructure cost + human review cost + expected failure-handling cost

For a monthly estimate:

estimated monthly cost = completed workflows × estimated unit cost + fixed operating costs

These are calculation templates, not forecasts. Each input should carry:

• its unit;
• its source;
• the date and version to which it applies;
• whether it is verified or assumed;
• a low, expected, and high case when variability is material.

The denominator also matters. Cost per API request can hide retries, multi-step flows, review, and failed completion. The product decision should use a unit aligned with business operations, such as a reviewed case or completed workflow, while clearly defining what “completed” means.

An AI path should not be approved on cost grounds until the team can compare it with the ordinary-software alternative using the same operational unit.

Privacy is an input map, not a checkbox

A privacy review begins by mapping data movement. For the proposed workflow, document:

1. data received from the user or business system;
2. data added through retrieval;
3. data sent to each external or internal component;
4. logs, caches, traces, and evaluation records created;
5. people or services permitted to access those records;
6. retention and deletion rules.

No claims about provider retention, training use, residency, or contractual protection can be made from the supplied sources. Those fields remain unresolved until supported by applicable primary documentation and organizational review.

A practical default is to block sensitive data from the prototype until its handling has been explicitly approved. This is a recommendation, not a statement about a particular provider’s controls.

A concise approval path

A product or architecture review can use the following sequence:

1. Write the user or business failure in observable terms.
2. Define the acceptance condition and unacceptable outcome.
3. Compare deterministic search, rules, forms, and workflow automation.
4. If AI is still justified, name the exact model version.
5. Attach primary sources for capability, price, context, and API behavior.
6. Build an evaluation set around representative and difficult cases.
7. Set scoring and stopping rules before examining results.
8. Name the human-review and escalation boundary.
9. Map sensitive data through every component.
10. Calculate cost per completed operational unit.
11. Record assumptions separately from verified facts.
12. Approve, revise, or stop the proposal.

The recommendation to pause is invalidated when the missing evidence is supplied, the evaluation method is agreed, the privacy and review boundaries are approved, and the economics can be calculated from version-specific inputs. Approval still depends on the resulting evidence; completing the form does not guarantee a positive decision.

The decision today

Based only on the supplied material, no specific AI model, RAG design, or agent workflow can be responsibly recommended. The sources may be relevant to later language-platform research, but they do not answer the product, model, privacy, evaluation, or economics questions required for AI feature approval.

That is a useful outcome. A disciplined pause prevents an implementation choice from outrunning its evidence. Define the problem, test ordinary software first, and require a versioned evidence record before committing to AI.

If you are reviewing an AI feature, automation proposal, or model-backed architecture, I can help structure a focused technical audit around its evidence, delivery risks, and operating constraints.

Official sources

• Python 3.12.14, 3.11.16 and 3.10.21 are now available — Python Insider
• Nicolai Parlog — Inside Java