Sampling Risk vs. Non-Sampling Risk: A Bigger Test Can Still Ask the Wrong Question

Audit confidence depends on both which items are examined and what the procedure actually establishes. Increasing coverage helps only if the test is capable of answering the audit question.

An original fictional retail case makes the distinction tangible. A retailer requires an authorized supervisor to approve a refund before it is released. The audit question is whether this approval operated for the relevant refunds in the period. The auditor first needs a reliable population, a clear approval rule and evidence that identifies both the approving role and the timing.

One process, two ways to reach the wrong conclusion

In the first scenario, the auditor defines the relevant refund population and applies an appropriate approval test to a properly selected random sample. By chance, the selected items contain no approval deviations, while deviations exist elsewhere in that population. A sample-based conclusion can therefore differ from the conclusion obtained by applying the same test to every item. This is the sampling-risk mechanism.

The scenario illustrates a possibility, not a recommended sample size or a quantified probability. A clean sample is evidence to evaluate within a sampling design; it is not proof that every refund complied.

In the second scenario, the auditor reads a field labelled Acknowledged by and treats any recorded name as evidence of supervisor approval. That field actually records the cashier who received the request. Even if the auditor checks every refund record, the procedure does not establish approval by an authorized supervisor before release. This is a non-sampling problem: the wrong evidence is being used to answer the question.

The second scenario is deliberately separate from the first. It isolates a different failure mechanism rather than claiming that one observed problem explains every audit error.

Match the response to the mechanism

For sampling risk, examine the population, selection method, sample size, observed results and the basis for extending a conclusion beyond the tested items. A larger, appropriately designed sample may reduce sampling risk; size alone does not repair an incomplete population or biased selection.

For non-sampling risk, inspect the procedure itself. Does the field represent the control being tested? Is the approver authorized? Is the timestamp the approval time or a later update? Can the evidence distinguish approval from acknowledgment? A walkthrough, corroboration and review of test instructions can expose this mismatch before the error is repeated across many records.

An automated full-population routine can repeat a mistaken rule consistently. Consistency of execution is useful, but it does not establish that the rule is valid.

Understand what full-population testing changes

When the same appropriate procedure is applied to the entire correctly defined population, the uncertainty arising from selecting only part of that population is removed for that procedure. Other limitations remain. Missing records, incorrect field mapping, weak evidence and mistaken interpretation can still affect the conclusion. Coverage of one approval attribute also says little about a separate question such as whether each refund related to a genuine sale.

This is why a statement such as all records passed needs context: all of which population, against which rule, using which evidence, for which period?

Try the comparison yourself

For a refund process you understand, write two short failure scenarios. In one, keep the test appropriate and vary the items selected. In the other, keep coverage unchanged and change the procedure or interpretation. Then identify a response that addresses each mechanism directly.

The distinction is consistent with ISA 530, particularly paragraphs 5 and A1, referenced in the IAASB 2026 Handbook. The standard concerns financial-statement audits. This original internal-audit illustration explains the concept; applicable engagement standards and local requirements must be considered separately.

The practical lesson: ask whether the evidence answers the right question before celebrating the number of records checked.

Inside Assurance — Beyond the numbers. Inside the business.