Continuous testing is not continuous assurance

Why continuous financial testing needs a control contract

On October 13, 2026, Workday announced planned early-adopter availability in early 2027 for an Accounting Agent that it describes as continuously testing financial data, policies, and controls. The direction matters: accounting software is moving from periodic assistance toward persistent monitoring and action. The timing and wording matter too. Planned early-adopter availability is not broad availability, a customer adoption measure, proof of effectiveness, or evidence that an auditor will rely on the output.

The practical problem exists regardless of vendor. A test can run every hour and still be weak. It may read an incomplete population, use a silently changed rule, suppress errors, mix retries with original transactions, or close an exception because a model produced a plausible explanation. Frequency amplifies both good design and bad design.

A defensible system therefore begins with six linked objects: a complete population, an approved deterministic rule, a resolved version, a durable exception, a named owner, and a retest. AI can make investigation faster, but it should not blur those objects or substitute a narrative for operating evidence.

authoritative population
  -> completeness reconciliation
  -> approved deterministic test + version
  -> immutable result per eligible item
  -> AI-assisted triage (advisory only)
  -> named-owner investigation and disposition
  -> remediation + retest
  -> independent review + release manifest

Define the test as a controlled object

A natural-language control description is not executable. Convert it into a contract that a qualified reviewer can understand and reperform. The contract names the risk and assertion, identifies every eligible record, states the expected condition, specifies deterministic logic and boundaries, and connects each exception to an owner and deadline.

Keep the deterministic rule distinct from AI interpretation. If the control is “every supplier-bank change requires independent approval before the next payment,” the pass/fail test joins authoritative change, approval, identity, and payment records. A model may summarize an exception or suggest the evidence to request. It should not infer that an approval “probably occurred” and turn a failure into a pass.

control_id: AP-BANK-004
population: supplier_bank_changes where effective_at in period
pass_when:
  approval.status == "approved"
  and approval.approver_id != change.requestor_id
  and approval.timestamp < first_payment.timestamp
fail_when: deterministic pass_when is false
error_when: any required source or identifier is missing
rule_version: ap-bank-004@3.2.1
ai_allowed: summarize supplied records; cluster root-cause candidates
ai_prohibited: change result; approve disposition; release payment
retest: rerun rule against corrected authoritative records

Prove population completeness before counting exceptions

Population completeness is the first control test. Start from the authoritative event or transaction source, not from the agent's successful-run table. Reconcile all eligible items through extraction, testing, investigation, remediation, and closure. Errors, duplicate events, failed calls, retries, reversals, manual completions, and records routed around the agent remain part of the story.

PCAOB AS 1105 is useful even when management is building the process rather than performing an audit. It emphasizes the accuracy and completeness of company-produced information and the reliability of its sources and transformations. For management, that translates into retained query parameters, extract timestamps, row counts, monetary totals, source ownership, transformations, and differences between stages.

eligible = source_events(period, entity)
tested = test_results(rule_version, run_id)

assert unique(eligible, "source_event_id")
coverage = eligible.left_join(tested, on="source_event_id")

report = {
  "eligible_count": count(eligible),
  "eligible_amount": sum(eligible.amount),
  "passed": count(coverage.result == "pass"),
  "failed": count(coverage.result == "fail"),
  "errored": count(coverage.result == "error"),
  "not_tested": count(coverage.result is null),
  "unexpected_results": tested - eligible
}
assert report.not_tested == 0
assert count(report.unexplained_differences) == 0

For automated controls, PCAOB AS 2201 also highlights an important dependency: benchmarking an automated control depends on effective IT general controls, no relevant program changes, and a continuing baseline. A monitoring rule cannot inherit trust forever. Source-system, report-logic, access, deployment, configuration, and threshold changes need explicit detection and evaluation.

Use AI after the test, not inside the conclusion

AI is most useful where exceptions are numerous and evidence is messy. It can group similar failures, extract dates and identifiers from supplied records, summarize conflicting facts, propose investigation questions, identify missing documents, and draft an owner-ready packet. Those tasks reduce clerical work without redefining the control result.

Every AI statement should be labelled as a fact from supplied evidence, a deterministic result, an inference, or missing information. Capture the provider, resolved model version, prompt/schema, retrieval sources, tools, and run ID when the output becomes part of the investigation record. A later model update must not silently rewrite an earlier disposition.

Also maintain a false-negative challenge. Sample passed items, seed known failures, compare relevant post-close adjustments or complaints, and monitor for patterns the current rule cannot express. A low exception count may mean a healthy process, an incomplete population, or a weak detector. The evidence must distinguish them.

Worked example: duplicate payments that disappear in retries

An accounts-payable team runs a daily duplicate-payment test. The initial dashboard reports 42,180 invoices tested, 37 exceptions, and 36 closures. AI-generated summaries look precise. The reviewer then compares the run with the authoritative invoice and payment-event tables. The source contains 42,236 eligible invoice records. Fifty-six items were absent because the workflow deduplicated retry events before preserving their lineage.

The missing records include two invoices paid under different vendor-master IDs after a merger. The deterministic rule did not see both candidates in the same population. This is not an AI summarization problem. It is a population and identity-resolution defect, so every downstream exception rate is unreliable until the population is rebuilt.

The corrected packet retains both rule versions, both populations, the change approval, the two new exception investigations, recovery status, and the independent retest. The control owner can then decide whether the daily control operated as designed and whether the defect has broader implications. AI can explain the sequence, but it cannot make that conclusion.

Failure modes to test deliberately

Stop conditions should be machine-visible and human-readable. A missing source, broken join, unexplained population gap, unauthorized access, unapproved rule version, unresolved material exception, or expired review must produce HOLD, not a green status with a footnote.

Run a 30-day pilot before expanding the loop

Week 1: choose one bounded, high-volume control with an authoritative population and no autonomous posting authority. Write the control contract, name owners, define stop conditions, and preserve one completed-period baseline.

Week 2: implement and validate the deterministic rule. Use positive, negative, boundary, duplicate, reversal, missing-field, late-arriving, and known-error records. Reconcile every eligible item to a result and independently reperform a sample.

Week 3: add AI triage for failed and ambiguous records only. Seed misleading explanations, contradictory evidence, absent support, sensitive fields, model fallback, prompt injection in source text, and a result the model wants to recategorize. Confirm the workflow preserves the deterministic result and stops when required.

Week 4: run end to end with Accounting, control owner, ICFR/SOX, ITGC, systems, security/privacy, and Internal Audit. Measure unexplained population differences, test coverage, false positives, known misses, exception age, owner response, retest cycle time, unauthorized overrides, version gaps, packet retrieval time, and reviewer rework.

Document the baseline before judging improvement: current test frequency, manual hours, exception volume, late closures, repeat findings, and evidence retrieval time. Report both labor saved and new review work created. Faster triage is useful only when completeness, detection quality, ownership, and closure remain visible.

Approve expansion only when the team can retrieve and reperform a complete packet without relying on the original operator's memory. Keep a parallel path for rule and source failures. Do not widen scope merely because the AI summaries are useful.

For evidence delivered to an auditor, continue with the AI audit evidence workflow. For transaction preparation and segregation of duties, use the journal entry review workflow. This guide focuses on the monitoring loop that connects complete populations to owned exceptions and retests.

FAQ

Is continuous control testing the same as continuous assurance?

No. Frequent tests produce more current evidence, but the conclusion still depends on control design, reliable data, complete coverage, test quality, exceptions, changes, ITGCs, and qualified judgment. External auditors independently determine their procedures and reliance.

Can AI determine whether a control is effective?

AI can organize evidence, compare supplied records, and draft questions. A named, competent person should determine the management conclusion under the organization's control framework. Deficiency severity, accounting impact, and audit conclusions remain outside the model's authority.

Should every control run continuously?

No. Frequency should match risk, transaction timing, source reliability, remediation capacity, and the control objective. A daily test that nobody can investigate may be worse than a well-designed monthly control with timely closure.

What if the rule changes during the period?

Freeze both versions, identify the affected population, validate the change, obtain approval, and decide whether prior items need rerunning. Do not overwrite the old result or present the period as if one rule governed it.

Does Workday's announcement prove continuous financial testing works?

No. Workday announced planned early-adopter availability in early 2027 for its Accounting Agent. Use the announcement as a signal to design governance now, not as evidence of adoption, operating effectiveness, return on investment, or audit acceptance.