To stop an AI agent from inventing values on an invoice, an order or a customer request, choosing a better model is not enough. You need to design the process around the model, requiring every value to come with the sentence of the document it was read from, leaving calculations and checks to fixed rules and sending anything without proven origin to a person.
This article explains why language models sometimes invent values, which practical rules reduce that risk and how to organise a process in which the AI proposes and the platform decides what is executed.
Why can an AI agent invent values?
A language model generates the most likely answer to the text it receives. In most cases that answer matches what is in the document. But when the document is badly scanned, shows two totals, mixes currencies or simply does not contain the value requested, the model may fill the gap with a plausible number. This is what is called a hallucination.
In an administrative process, a plausible but wrong value is worse than a missing one. A swapped total on a supplier invoice, a wrong quantity on an order or an invented bank account number go unnoticed if nobody checks them against the original document.
- Ambiguous documents. Invoices with discounts, carriage and subtotals offer several candidate numbers.
- Missing information. An email asking for «the same as last time» does not state quantities.
- Asking the model to go beyond reading. When the AI is asked to add up, convert or check, the room for error grows.
How do you require the origin of every value?
The most effective rule is simple to state. No value enters a system without the sentence of the document it was taken from. The agent returns the value and the quotation, and the platform confirms that the sentence really exists in the email or the document.
- The agent reads the document and proposes each field, such as the supplier, the invoice number, the lines and the total.
- For each field, it gives the exact sentence where it found it.
- The platform looks for that sentence in the original document.
- If the sentence does not exist, the value does not go through and the case goes to a person.
This check does not depend on the goodwill of the model. It is done outside the AI, by a rule that always behaves the same way, and it makes human review much faster, because whoever decides can see straight away where each number came from.
Which work should stay outside the AI?
AI is good at reading messy text. It is not the right tool for arithmetic or for checking values. So, in a well designed architecture, the AI reads and the platform calculates.
| Task | Who should do it |
|---|---|
| Interpreting an email or a document | The AI, with the source sentence for each value |
| Adding up lines, applying discounts and calculating taxes | Platform rules, without AI |
| Matching the invoice with the order and the goods receipt | Platform rules, without AI |
| Matching bank movements to invoices | Platform rules, without AI |
| Checking calendar availability | Platform rules, without AI |
| Deciding a doubtful case | A person in the team |
Business rules, such as tolerances, trusted suppliers or value limits, should be managed in versioned sets. That way, the history of each case shows which rule was applied, and a change tomorrow does not alter what was decided yesterday.
How do you limit what the agent can execute?
Even with correct values, the next question is what the agent can do with them. The safe answer is that the AI executes nothing on its own. Only the actions the organisation has declared for each system are executed, in the published version of each process.
- Declared actions. Creating an order in the ERP or replying to a customer are configured steps, with no code or free commands generated by the model.
- Value limits. A rule such as «above this amount, ask for approval» suspends the case before the step until a person approves.
- Writes that never repeat on their own. If the process is interrupted halfway, a person decides whether the write has already happened, instead of the system trying again and duplicating the record.
Imagine a distributor that receives orders by email. The agent reads an order for 40 boxes and quotes the line with the quantity, the platform calculates the total from the price list and, because the amount exceeds the limit set by the organisation, the case waits for the sales manager to approve it before it enters the ERP.
When should a case go to a person?
The aim is not to remove human involvement but to use it where it makes a difference. A case should go to a person when the origin of a value is missing, when the interpretation is uncertain or when a limit set by the organisation requires it.
At those moments, the person should receive the case with what has already been done and the decision still needed, not a raw document to start again from scratch. Every human decision stays in the case history, with the change and the reason, and the team's corrections go on to guide the cases that follow.
How does EngiAIFlow apply these rules?
In EngiAIFlow, the AI proposes and the platform decides what is executed. Every value the AI extracts from an email or a document comes with the sentence where it was read, and the platform confirms that the sentence exists. Without a sentence, the value is routed to a person. Calculations, calendar availability, bank reconciliation and invoice checking are done by the platform, without AI.
Only the actions declared for each system are executed, value limits suspend the case until there is approval, and a write to a connected system is never repeated automatically. Each case is completed with the version of the actions and rules it started with, and the history records what was done at each step and every human decision.
Frequently asked questions
What is an AI model hallucination?
It is a plausible answer with no basis in the text received, for example a total or a quantity that is not in the document. It happens mostly with ambiguous or incomplete documents.
Does a newer model stop inventing values?
It reduces the risk but does not remove it. Effective protection lies in the process around the model, such as requiring the origin of each value and confirming it exists in the document.
Why should calculations not be done by the AI?
Because fixed rules always give the same result for the same data. The AI should read and propose, while totals, taxes and checks should be done by the platform.
Does human review cancel out the gain from automation?
No, as long as only cases without proven origin, with doubts or above a limit reach a person. The rest go through on their own, and the person receives the case already prepared.
What happens if a process stops in the middle of a write to the ERP?
In EngiAIFlow, a write to a connected system is never repeated automatically. A person decides whether the write has already happened, which prevents duplicate records.