nvsbl

Operations / Field notes

Custom AI systems for operations teams: what to define before you build

Custom AI is worth investigating when an important workflow has requirements your existing tools cannot meet, the team can provide reliable inputs, and someone can judge whether the result is useful. Begin with a specific piece of work. The decision to build should follow evidence about that work, not a wish to add an agent to the business.

A custom AI system combines software built for a particular workflow with AI for selected tasks, such as interpreting a request or drafting a summary. The surrounding software still needs to decide which records may be read, who can authorize an action, where results belong and what happens when something fails. Those decisions are part of the system you would commission.

Choose a workflow you can explain from start to finish

“Our information is scattered” describes a frustration. “Prepare the handoff for an incoming service request” describes work you can examine. For that handoff, identify the person waiting for it, the information they need and the decision they make next.

Separate the parts of the problem. Finding a record, interpreting an ambiguous message and updating a schedule require different checks. An AI-generated summary might help with interpretation while a simple lookup supplies the authoritative schedule. The team should decide where interpretation is useful and where an exact rule is required.

This focus on context has a useful reference point: NIST's voluntary AI Risk Management Framework asks teams to understand an AI system's purpose, setting and potential impacts before making a go/no-go decision. It does not prescribe one universal checklist. The brief below is our practical starting point for an operations discussion. NIST AI RMF Core

Write a one-page workflow brief

Fill this in before asking for an architecture or a price. An unresolved answer is valuable: it identifies a discovery task rather than hiding it inside the build.

Field Question to answer Example answer for a hypothetical service team
Trigger What starts the work? A service request enters the shared inbox
Authoritative records Which source wins when information conflicts? The approved work-order record owns the address and appointment
Intended output What does the next person need? A draft handoff with the request, relevant record links and missing information
Decision What judgment is involved? An operator decides whether the request can be scheduled
Permissions What may the system read or change? Read approved records; draft a handoff; do not change appointments
Exception owner Who handles an uncertain result? The dispatcher reviews unmatched or conflicting records
Stop condition When must processing pause? The request cannot be linked to one verified work order
Success evidence What would justify continuing? Operators can complete the handoff with acceptable checking effort and fewer unresolved omissions

For each record, name its owner and how access could be granted. “It is in someone's email” is not yet a data-access plan. Also ask whether the proposed reader should be allowed to see every field in that source.

Walk through an example, including the awkward cases

In this hypothetical workflow, a customer asks to move a service visit. A proposed system would find the matching work order, show the current appointment and draft a handoff for the dispatcher. The dispatcher would make the scheduling decision. No appointment would change merely because the draft sounded convincing.

Now change the input. The customer gives an old address. Two work orders match. The schedule was updated after the message arrived. In each case, the useful output might be a request for clarification rather than a completed handoff. Specify that behavior before choosing a model or an integration.

Start a trial with examples the team is authorized to use. Keep examples used to improve the system separate from examples reserved for evaluating it. Ask operators to record wrong matches, missing information, corrections and time spent checking. A polished demonstration is insufficient evidence for changing the real scheduling process.

Decide whether to configure, investigate or wait

Configure an existing tool when it can support the workflow and its required controls with an acceptable operating burden. The problem may be an unused feature, inconsistent records or an unclear process owner.

Investigate a custom build when a specific requirement remains unmet and you can test whether a proposed design addresses it. Examples include a review screen tailored to a particular decision or reconciliation across records with company-specific rules. These are reasons to investigate, not proof that custom software will be better.

Wait and clarify when nobody agrees which record is authoritative, access is unavailable, or nobody owns exceptions. A prototype can still help explore the problem, but it should not quietly become the operating process.

Make the next conversation concrete

Bring the brief, a walkthrough of current work and a few redacted examples. Ask a prospective builder to identify assumptions, explain the smallest useful trial and name the evidence that would make them recommend stopping. Include the person who would operate the system after delivery.

Invisible Product Inc. offers custom engineering engagements around a team's own business data. We can discuss the workflow and what a proposed build would need to establish. Discuss a workflow.

If the workflow is already clear, use the no-code versus custom AI comparison to assess approaches, then the scoping checklist to prepare acceptance scenarios.

Sources and authorship

Prepared by Invisible Product Inc. The service-team example and worksheet are original, hypothetical teaching material, not a customer case study or a claim about a delivered system. External reference checked September 27, 2026: NIST AI Risk Management Framework Core.