Deterministic AI in Accounting: Why Replay Matters

Ask a language model the same question 1,000 times at temperature 0 and you can get 80 different answers. That is fine for a first draft of an email. It is a problem for a number your auditor will ask you to reproduce in February.

TL;DR: Deterministic AI gives the same output every time it sees the same input, so you can replay a past result and get the same answer. Probabilistic AI, including large language models, can vary between runs. In accounting, use probabilistic AI to read and suggest, and make the step that changes the ledger deterministic and replayable.

What is deterministic AI in accounting?

Deterministic AI in accounting is AI whose ledger-affecting output can be rerun on the same inputs and produce the identical result.

Probabilistic AI is AI that produces its output by estimating likely answers, so the same input can give a different output on a different run.

Most of what is called AI in finance tools today is a mix of the two. A model reads a bank memo or an invoice PDF and suggests something. A rule, query or formula then decides what gets matched, coded or posted. Where that line sits is the most important design question you can ask a vendor, and most product pages do not answer it.

A May 2026 survey of determinism in financial AI by researchers at Amazon separates three ideas that often get blurred (arXiv 2605.23955):

• Computational determinism: bit-exact identical outputs given identical inputs, model weights and hardware.
• Reproducibility: outputs consistent enough to support the same downstream decision, even if the wording differs.
• Auditability: any algorithmic decision can be reconstructed and independently verified after the fact.

For a controller, the third one is the goal. The first two are ways of getting there.

Why isn't AI deterministic by default?

Two reasons. The first is by design: language models pick each next word from a probability distribution, and settings like temperature control how much randomness goes into that pick. The second surprises people: even with temperature set to 0, outputs can still change.

In September 2025, Thinking Machines Lab ran the same prompt through a model 1,000 times at temperature 0 and got 80 unique completions. The runs matched for the first 102 tokens and then split (Thinking Machines Lab, Sept 2025). Their explanation: server-side batching changes the order of floating-point math, so your request's result depends on what else was running at the same time.

Vendors say the same thing in their own docs. OpenAI's guide to its seed parameter promises "(mostly) consistent outputs" and states that "determinism is not guaranteed" (OpenAI Cookbook, Nov 2023). Models also get updated and retired. A prompt that ran on one model version in September may not have that version available to rerun next year.

None of this makes language models useless in finance. It means a model call is not, by itself, something you can replay.

Deterministic vs probabilistic AI: what is the difference for a finance team?

• Same input, run twice: deterministic gives the same output every time; probabilistic can differ, even at temperature 0.
• How it decides: deterministic follows a written rule, query or formula; probabilistic uses learned patterns and sampling.
• How it is explained: deterministic shows the rule that actually ran; probabilistic gives reasoning written after the fact.
• How you test it: deterministic, rerun and diff; probabilistic, run many times and measure the variation.
• What it is good at: deterministic handles matching, calculations, thresholds and posting; probabilistic handles reading documents, classifying messy text, drafting commentary and suggesting rules.
• Audit evidence: deterministic output can be reperformed by an outsider; probabilistic output needs the stored result plus evidence of human review.

The right column is not the bad column. It covers work rules handle badly, like reading a vendor name off a scanned invoice. The question is which column your ledger depends on.

Why does replay matter to auditors?

Replay is rerunning a past AI result with the same inputs, rule versions and settings, and getting the same answer or a logged reason why not.

Auditors already work this way. PCAOB AS 1105 lists reperformance, the independent execution of procedures or controls originally performed by company personnel, as a way to gather audit evidence (PCAOB AS 1105). AS 1215 requires audit documentation that lets an experienced auditor with no previous connection to the engagement understand "the nature, timing, extent, and results of the procedures performed, evidence obtained, and conclusions reached" (PCAOB AS 1215).

Those standards bind your auditor, not you. But they shape what your auditor will ask for. If a September match exists in your ledger, someone may ask in February why it exists. "The model decided" is not reperformable. "Rule v14 ran on the September 30 snapshot and matched on amount, date within 3 days and invoice number in the reference" is.

COSO's February 2026 guidance on internal control over generative AI names "model drift" and "frequent configuration changes" as risks, and notes that GenAI "can be confidently wrong" (COSO via PR Newswire, Feb 23 2026). Replay is how you catch drift: if the same inputs now give a different answer, something changed, and you need to know what.

We covered the auditor's side in more depth in 9 questions your auditor will ask about AI reconciliation.

Want every match to come with the source rows, the rule and the SQL behind it? Join the Yoraito waitlist for early access.

How do you make an AI-assisted process replayable?

You do not need a deterministic model to have a replayable process. You need to store what the model said, and make every step after it deterministic.

That is the whole design. When a model extracts an invoice total or suggests a match, save its raw output as a record. From then on, the match, the coding and the entry come from versioned rules or queries that read the stored record. A replay reads the stored model output instead of asking the model again, so model randomness never reaches the ledger.

To make that work, each AI-assisted result needs a replay record:

• 1. Input snapshot ID: a frozen copy of the source rows the AI saw, not a live table.
• 2. Rule or query version: the exact logic that produced the result.
• 3. Model name and version: plus the provider's fingerprint if it offers one.
• 4. Prompt or template version: the instructions the model received.
• 5. Parameters: temperature, seed, thresholds and tolerances.
• 6. Raw model output: stored as returned, before any cleanup.
• 7. Final result and a hash of it: so a replay can prove a match, not just look similar.
• 8. Reviewer, decision and timestamp: who approved it and when, which is what AS 1215 asks auditors to be able to see for their own work.

If you are drafting vendor questions, start with fields 1, 2 and 6, since they are what make a replay possible at all. Our 12 audit trail questions for your AI accounting RFP cover how to ask for them.

What does a replay look like in practice?

Take a September bank reconciliation. In February, your auditor picks bank line BT-48211 for $12,480.00, matched to invoice INV-1093, and asks how the match was made.

If the match came straight from a model call, you can rerun the prompt, but you might get a different match or a different explanation, and you cannot prove which one ran in September.

If the match came from a versioned rule over a frozen snapshot, you rerun the rule and diff it against what was booked:

-- Replay rule v14 on the Sept 30 snapshot and compare to what was booked
WITH booked AS (
SELECT bank_txn_id, ledger_entry_id
FROM match_results
WHERE period = '2026-09' AND rule_version = 'v14'
),
replay AS (
SELECT b.bank_txn_id, l.ledger_entry_id
FROM bank_lines_snapshot b
JOIN ledger_open_snapshot l
ON l.snapshot_id = b.snapshot_id
AND l.amount = b.amount
AND ABS(l.entry_date - b.txn_date) <= 3
AND b.reference ILIKE '%' || l.invoice_no || '%'
WHERE b.snapshot_id = 'S-2026-09-30'
)
SELECT COALESCE(k.bank_txn_id, r.bank_txn_id) AS bank_txn_id,
k.ledger_entry_id AS booked_match,
r.ledger_entry_id AS replayed_match
FROM booked k
FULL OUTER JOIN replay r ON r.bank_txn_id = k.bank_txn_id
WHERE k.ledger_entry_id IS DISTINCT FROM r.ledger_entry_id;

Zero rows means the replay agrees with the books. Every row returned is a difference that needs a logged reason: a manual override, a corrected input, an approved rule change. The auditor can read the query, and with access to the snapshot, run it themselves.

We walk through what each match record should hold in AI bank reconciliation: what every match should explain.

Where should probabilistic AI sit in the close?

Upstream of the ledger, with a stored output and a reviewer. Good fits:

• Reading invoices, receipts and bank memos into structured fields.
• Suggesting a GL code or a match for a human or a rule to confirm.
• Drafting flux commentary that a reviewer edits and signs.
• Proposing a new matching rule, which a person approves and versions before it runs.

Poor fits are anything where the model's answer is the final answer: posting an entry, clearing a match, calculating an accrual or deciding an exception is resolved, with no rule or reviewer in between.

One caution. Deterministic does not mean correct. A bad rule gives the same wrong answer every time. Replay proves consistency, not accuracy, which is why review thresholds and exception queues still matter. Our post on why AI journal review needs governance, not just automation covers that side.

FAQ

Are large language models deterministic?

Not by default. They sample each next word from a probability distribution. Setting temperature to 0 reduces variation but does not remove it: Thinking Machines Lab got 80 unique answers from 1,000 temperature-0 runs. OpenAI's own docs say determinism is not guaranteed even with a fixed seed.

What is the difference between deterministic and probabilistic AI?

Deterministic AI returns the same output for the same input every time, because it follows fixed rules or calculations. Probabilistic AI estimates likely answers, so outputs can vary between runs. Finance tools often combine both: a model suggests, a rule decides.

Can you use generative AI in a SOX environment?

Many teams do, with controls around it. Store the model's raw output, keep a human reviewer on anything that affects the ledger, and make the posting step a versioned rule. COSO's February 2026 guidance treats GenAI as a risk to control, not a reason to ban it. Confirm the approach with your auditor.

What does replay mean in accounting AI?

Replay means rerunning a past AI-assisted result with the same inputs, rule versions and settings, and getting the same answer or a logged reason for any difference. It is how you answer an auditor who asks why a specific match or entry exists months later.

Does deterministic AI mean accurate AI?

No. A deterministic rule that is wrong is wrong every time. Determinism makes errors repeatable and findable, which is useful, but accuracy still depends on rule design, testing and review. Run a pilot on a closed period and check results against what your team booked.

Check one past result this week

Pick one AI-assisted match or entry from last month and try to replay it: same inputs, same logic, same answer. If you cannot, list which of the eight replay fields were missing, and add them to your vendor questions or your pilot plan. Our 30-day AI accounting pilot scorecard has a slot for this. Yoraito is being built so finance teams and auditors can check AI output instead of trusting a black box. To hear when Yoraito opens up, join the waitlist.