Did Orders Missing Approval Signatures Really Decrease?

Did the number of orders without approval signatures get better, worse, or is it too early to tell?

The audit question

You know how many orders used to slip through without approval signatures. After a fix, you count again. Is the improvement big enough to be real — or could it just be luck?

Scenario

During a routine review, an internal auditor keeps finding purchase orders processed without the required approval signature. Management responds with a new sign-off policy and staff retraining. A few months later comes the real question: did it work? The auditor takes ten counts of non-compliant orders from before the change and ten from after, and runs both through the tool.

Data

Baseline (data0)
7
3
11
2
15
5
9
1
18
4

New Process (data1)
2
1
4
1
6
2
3
1
7
1

What the tool returned

The auditor pasted both datasets into the tool and ran the analysis. The following result came back:

Analyser

🖊️

Before the change (data0), the metric averaged 7.5, typically ranging between 1.0 and 18.0. Since the change (data1), it averages 2.2. With 94% confidence, the true figure now sits between 1.5 and 2.9. Every sample tested — 100% — confirmed a reduction of approximately 5.3, giving good assurance that the decrease is real, not a random fluctuation.

Size of the change

−5.3

Verdict

Strong decrease

Model: Poisson (auto-detected) n₀ = 10 · n₁ = 10 · ESS = 409

Audit conclusion

Comparing order compliance before and after the mandatory sign-off policy shows a clear drop in orders processed without an approval signature. The fix worked. The control gap identified earlier is considered resolved, and the finding can be closed — provided the same types of orders and the same approval process were in place during both measurement periods.

Tool usage benefits

Closing a finding on "the average looks lower" is a judgement call. Closing it on a quantified result from ChangeVerifier — how large the change is, how certain it is, and where the true value most likely sits — is evidence. If a regulator or external reviewer later asks why the finding was closed, the answer is already in the working papers. From pasting the data to reading the verdict took under a minute, with no statistics background needed.