AI Feature or Ordinary Software: Use an Evidence Gate

Make the AI feature decision before choosing a model

A roadmap item arrives labeled “AI,” but the label does not identify the user problem, technical mechanism, or business case. The immediate decision is not which model to integrate. It is whether the proposed outcome requires a model at all.

That distinction matters because choosing an AI architecture creates questions that ordinary software may not create in the same form: which model version is being evaluated, what its supplied API contract permits, where human review is required, what data may leave the application boundary, and how variable usage becomes a per-task cost. Those questions cannot be resolved with enthusiasm or a generic reference to AI capability.

The supplied sources create a useful constraint for this decision. They identify current Python releases and a general Open Web documentation resource, but they do not provide primary evidence about any AI model, model API, pricing schedule, context limit, or capability evaluation. The responsible recommendation is therefore narrow: use an evidence gate before approving a model-dependent implementation, and keep ordinary software as the default candidate until the missing evidence is supplied.

Official sources

• Verified fact: Python Insider announces Python 3.14.7 and Python 3.13.15 as available releases. The supplied metadata identifies Python Insider as the official blog of the Python core development team. It does not provide release-detail evidence beyond the announcement title, so no compatibility, performance, or behavioral claim is inferred here.
• Verified fact: MDN Web Docs describes documentation for Open Web technologies, including HTML, CSS, and APIs for websites and progressive web apps. The supplied metadata does not identify a particular browser API or implementation pattern, so none is assumed.

Evidence gap: Neither supplied source establishes the capability, price, context limit, or API behavior of an AI model. No model recommendation or model-specific cost estimate can therefore be verified from this evidence set.

That is not a minor editorial limitation. It is the central product constraint.

Turn the requested feature into a testable problem

Before comparing implementations, rewrite the request without the word “AI.” A useful problem statement contains five parts:

1. User: Who is trying to complete the task?
2. Input: What information is available at decision time?
3. Output: What must the product return, change, or route?
4. Acceptance criteria: How will a reviewer determine whether the output is usable?
5. Failure cost: What happens when the output is wrong, incomplete, delayed, or exposed to the wrong party?

For example, “add AI to support intake” is not testable. A hypothetical, testable version might be: “Given a submitted support form, assign one category from an approved list, identify missing required fields, and send uncertain cases to a person before any customer-facing action.”

This example is an assumption for illustration, not a claim about an existing product or result.

Once the request is stated that way, the team can examine whether explicit validation and routing rules satisfy the acceptance criteria. If they do, ordinary software remains a credible implementation. If they do not, the unresolved cases become the candidate scope for a model-assisted path.

This decomposition also prevents a broad model integration from absorbing tasks that are already deterministic. Required fields, allowed values, permissions, and fixed routing conditions can be evaluated separately from any task that genuinely needs less structured interpretation.

Compare three implementation paths, not two

The decision is often framed as “AI or no AI.” A more useful comparison has three options.

1. Ordinary software

Use explicit validation, conditions, state transitions, and human-owned exception handling.

Choose this path when the task can be expressed through stable rules and the rules meet the agreed acceptance criteria. Its main advantage in this framework is not a universal claim that it is cheaper or faster. The advantage is that its behavior and test surface can be specified without relying on model evidence that is currently missing.

Reject this path if a representative evaluation shows that the required distinctions cannot be encoded or maintained within the team’s constraints.

2. A constrained model-assisted step

Use a model only for the unresolved portion of the workflow, while ordinary software controls input validation, permissions, allowed outputs, logging, and escalation.

This option should not be approved from the supplied sources alone. Approval requires a primary source for the exact model version covering relevant capability, API behavior, context limits, and pricing. It also requires task-specific evaluation rather than a general benchmark.

The model’s output should be treated as a proposal wherever the failure cost requires review. The surrounding application—not a human-like description of an “agent”—should own the workflow state and authority boundaries.

3. Defer the implementation decision

Deferral is appropriate when the problem is clear but the evidence is not. The next deliverable is then an evidence plan: identify candidate versions, collect their primary documentation, define the evaluation set, and obtain the data-handling review needed for the intended inputs.

Deferral does not mean abandoning the opportunity. It prevents a team from hardening an architecture around unsupported capability or cost assumptions.

A hypothetical support-intake implementation

Consider the illustrative support-form problem. The technical path can be divided into stages without assuming any particular framework, model provider, or browser API.

Stage 1: Deterministic intake

The application checks whether required fields are present, whether values belong to allowed sets, and whether the requester has permission to submit the data. These checks should remain ordinary software because their desired outcomes are explicit.

Stage 2: Rule-based routing

Known product areas, account states, or request types can map to approved categories when the mapping is unambiguous. The evaluation question is simple: do the rules classify enough cases correctly under the agreed acceptance criteria and maintenance budget?

No percentage can be supplied here because there is no task data in the source metadata. The threshold must be chosen by the product owner and operators according to the cost of incorrect routing.

Stage 3: Candidate model scope

Only cases unresolved by the first two stages become candidates for model-assisted classification. Before implementation, the team must add evidence for an exact model version and its API. The test should use representative, appropriately handled examples from the intended workflow.

The output contract should be narrow: an allowed category, an uncertainty or abstention state if supported by the designed workflow, and no authority to trigger irreversible action.

Stage 4: Human-review boundary

A person reviews cases that fail validation, fall outside allowed outputs, or carry a failure cost above the organization’s chosen threshold. The product team must define that threshold; it cannot be derived from the supplied sources.

Human review is not a decorative fallback. It affects queue design, operator workload, response time, and cost. If the review queue cannot be staffed or measured, the model-assisted path is incomplete.

Stage 5: Controlled release

Compare the candidate path with the rule-based baseline on the same evaluation set. Record errors by category rather than relying only on an aggregate result. Release only the scope that meets the predeclared criteria, and preserve a way to disable or bypass the model-dependent stage.

This path is an interpretation and proposed engineering method, not a report of a deployed system.

Evaluate the task rather than borrowing a benchmark

A decision-quality evaluation begins with examples that represent the intended inputs and failure modes. The set should be separated from examples used to adjust prompts, rules, or output handling.

For each candidate path, record:

• whether the output matches an approved result;
• whether the output violates the allowed structure;
• whether the case should have been escalated;
• whether sensitive input was handled according to the approved boundary;
• how much human review the path requires;
• whether response behavior remains acceptable for the product workflow.

The acceptance threshold is an assumption to be supplied by the business and product owners. A low-risk internal suggestion may permit a different threshold from an action that changes customer data or communicates externally.

The recommendation is invalidated if the ordinary path fails the criteria and a model-assisted candidate, supported by primary documentation and task-specific testing, meets them within the accepted privacy and cost boundaries. Conversely, the AI option should be rejected or redesigned if it cannot beat the baseline on the criteria that matter, requires an unstaffable review queue, or lacks verifiable provider documentation.

Calculate unit economics only after pricing is verified

There is no model pricing source in the supplied evidence, so a monetary estimate would be fabricated. The useful output at this stage is a calculation structure.

Calculation method:

AI cost per completed task = model API cost + retry cost + review cost + failure-handling cost + allocated integration and monitoring cost

Ordinary-software cost per completed task = allocated implementation and maintenance cost + exception-review cost + failure-handling cost

The terms require local inputs. For a model-dependent path, API cost must be calculated using the exact model version, supplied billing units, and primary pricing source. Retry frequency and review volume must come from the task evaluation, not from an unrelated benchmark.

This comparison may show that either path is preferable under explicit criteria. It may also show that the economics are too uncertain to approve either estimate. Uncertainty should remain visible rather than being replaced with a precise-looking number.

Put privacy and authority boundaries in the design

Before sending any production input to an external model API, the team needs an approved account of what the input contains, where it may be processed, what may be retained, and what must be removed or transformed. The supplied sources provide none of those provider-specific facts.

Until primary documentation and an appropriate privacy review are available, assume that sensitive production data is out of scope for a model evaluation. That is a conservative design assumption, not a statement about any provider.

Authority should be similarly explicit. A candidate model output can be advisory while ordinary software enforces permissions and allowed transitions. Any expansion from suggestion to action needs a separate failure analysis and review decision.

The approval checklist

Approve ordinary software when:

• the problem can be represented with explicit inputs, rules, outputs, and exceptions;
• the baseline meets the task-specific acceptance criteria;
• its maintenance and review costs fit the operating model;
• no unsupported AI dependency is needed to deliver the user outcome.

Approve a constrained model-assisted evaluation—not automatic production use—when:

• the deterministic baseline has a measured, material gap;
• an exact model version has primary documentation for capability, pricing, context limits, and API behavior;
• a representative evaluation set and acceptance threshold exist;
• privacy review covers the intended inputs;
• human-review ownership and capacity are defined;
• unit economics include API, retries, review, failures, and ongoing operation.

Defer when any required evidence is missing and the missing item could change the architecture, risk, or economics.

Evidence before architecture

The available sources support Python release awareness and general Open Web documentation. They do not support selecting or pricing an AI model. The sound decision is therefore not to improvise a provider comparison. It is to define the user problem, test the ordinary path, isolate any unresolved task, and require primary evidence before adding a model dependency.

An AI feature earns its place by meeting explicit product, technical, privacy, review, and economic criteria. Until then, ordinary software is not a lesser choice. It is the baseline against which the additional dependency must be justified.

If you are weighing a similar roadmap decision, a focused technical audit can help turn the request into an evidence plan, evaluation boundary, and implementation choice without committing prematurely to either architecture.