When Benford’s law applies, and when it does not

What the law says

In many collections of measured or counted quantities the leading digit is 1 about 30% of the time and 9 under 5% of the time. Simon Newcomb noticed it in 1881 from the wear on logarithm tables; Frank Benford (1938) checked it on twenty sets of numbers, from river areas to street addresses, and gave the expected share of each first digit d as log₁₀(1 + 1/d). The same reasoning gives the shares of the second digit and of the first two digits together.

The pattern appears when numbers are spread over several orders of magnitude and arise from growth, multiplication or the mixing of many different sources. Hill (1995) showed that picking numbers from a random mix of distributions leads to it. It is a property of the data-generating process, not a rule that every honest number obeys.

Data that usually fits

• Wide-ranging amounts. Values running from tens to hundreds of thousands: at least three or four orders of magnitude.
• Naturally occurring transactions. Accounts payable, sales invoices, expense lines across a whole company, journal entries, claims, inventory values, populations, areas.
• Many records. Thousands rather than dozens. Nigrini’s ranges were set on files of that size.
• No built-in minimum or maximum. Nothing in the process stops amounts at a ceiling or floor.

Data that does not, however honest

• Assigned numbers. Invoice and cheque numbers, account and employee IDs, phone numbers, postcodes, FIPS codes. They label things; their digits come from a numbering scheme. The sample file has a fips column: try it and watch it fail.
• Prices set by people. 9.99, 49.00, 199.00. Psychological price points and list prices pile up on a few leading digits.
• Capped or floored values. Meal allowances up to 50, a reimbursement ceiling, a minimum fee, a statutory amount. The cap moves the digits by design.
• A narrow range. Salaries within one grade, hourly rates, adult heights, a product’s unit price. If nearly everything sits between 300 and 900, the first digit cannot follow Benford.
• Small samples. With a few hundred amounts, chance alone moves the proportions a lot. The tool shows, under each test, the MAD a perfectly Benford sample of the same size would still have; when that is close to Nigrini’s ranges, the verdict is weak.
• Filtered extracts. “All payments over 10,000” or one vendor’s invoices change the range and the mix the law depends on.
• Rounded figures and estimates. Budgets, accruals and forecasts are often round on purpose. The last-two digits test shows this, and it is not a sign of anything improper by itself.

Reading the results

Start with the MAD verdict

The mean absolute deviation is the average gap, over all digits, between the share observed and the share Benford expects. Nigrini (2012, Table 7.1) set four ranges for it, from close conformity to nonconformity, from experience with many real files. The MAD is used instead of the chi-square because the chi-square grows with the number of records: on a file of 100,000 payments it rejects for differences too small to matter (Kossovsky 2021). The tool still shows the chi-square and the Kolmogorov–Smirnov statistic, with that caveat.

Then the digits

The MAD is an average and can hide a sharp spike. In the test suite, 400 invented amounts between 4,800 and 5,000, mixed into 20,000 genuine ones, leave the first-two digits MAD in close conformity, yet digits 48 and 49 have Z-scores of about 11 and 17. Look at every digit marked as over-represented at the 1% level (Z above 2.576). With 90 digits tested at once, one or two marks happen by chance; a cluster or a very large Z is worth a look.

Then the rows

Tap the digit. The tool lists the most frequent amounts in it, the largest ones and the first rows, and exports all of them with their original columns. Typical findings, roughly from most to least common:

• one amount repeated many times: a monthly fee, a standard rate, a recurring transfer;
• a policy threshold: amounts just under an approval limit or a tax threshold, which may be legitimate scheduling or deliberate splitting;
• a duplicate payment of the same invoice;
• a batch keyed or estimated by one person;
• and occasionally an amount that was made up.

The number duplication test lists repeated amounts directly. The summation test catches a different thing: a few very large amounts that dominate the total without changing the digit counts.

Negative amounts and amounts below 10

Credits and refunds come from a different process than payments, so they are tested on their own or, if you prefer, folded in as absolute values. Small amounts are usually immaterial and often fixed charges; by default they are left out, as in Nigrini’s practice. Both are switches on the tool page.

Conformity is not innocence

A file that fits Benford can still contain fraud: one inflated payment, a fictitious vendor billing ordinary-looking amounts, or invented numbers drawn to look natural. Digit tests are one screen among several, alongside the duplicate, threshold and outlier checks auditors already run.

Sources

• Benford, F. (1938). The law of anomalous numbers. Proceedings of the American Philosophical Society, 78(4), 551–572.
• Hill, T. P. (1995). A statistical derivation of the significant-digit law. Statistical Science, 10(4), 354–363.
• Kossovsky, A. E. (2021). On the mistaken use of the chi-square test in Benford’s law. Stats, 4(2), 419–453.
• Newcomb, S. (1881). Note on the frequency of use of the different digits in natural numbers. American Journal of Mathematics, 4(1), 39–40.
• Nigrini, M. J. (1996). A taxpayer compliance application of Benford’s law. Journal of the American Taxation Association, 18(1), 72–91.
• Nigrini, M. J. (2012). Benford’s Law: Applications for Forensic Accounting, Auditing, and Fraud Detection. Hoboken, NJ: Wiley.

Every formula and constant the tool uses, and how it was checked: Method and sources.