Measurement, counterfactuals and verification

An indicator needs a baseline and the possibility of being wrong

The most attractive impact figures are often the least informative: images processed, hectares covered, nominal users reached or funding 'mobilised'. They may describe production and reach; they rarely settle causal contribution. Measurement is useful when an investment committee can use it to decide whether to continue, redesign or discontinue an intervention.

Two counterfactual questions matter. What would users have experienced without the company's service? What would the company have achieved without this particular financier? The first concerns enterprise-level effects; the second concerns investor contribution. A service can improve users' outcomes even when a different bank would have financed its development on the same terms [R05][R06].

A practical evidence hierarchy

For a simple delivery milestone, dated records may suffice. For adoption, reliable administrative traces and user verification are necessary. For an investor-contribution claim, a credible historical counterfactual, independently corroborated decisions and explicit assumptions become important. Claiming monetised avoided losses or social welfare gains requires a still stronger comparative design, uncertainty ranges and sensitivity analysis.

The dossier proposes four operational assurance levels—not a certified standard: (1) verified outputs; (2) observed outcomes; (3) plausible contribution corroborated by several sources; and (4) causal identification through an appropriate research design. The standard of proof should rise with the materiality of the claim and the consequences of being wrong.

A ten-field indicator dictionary

Every indicator should identify its unique name, unit, observable event, geographical and temporal boundary, baseline, target and alert threshold, source and owner, frequency, population breakdown, and limitations plus verification method. In space applications, the record should additionally include sensor conditions, acquisition time, latency, model version, ground-truth quality and unequal error rates across locations.

A model can have excellent average accuracy and systematically fail communities with sparse reference data. A new warning system may shorten information delivery by an hour; the measurable outcome is faster availability or action, not an automatic financial valuation of avoided disasters. Monetisation also needs the severity and probability of events, decision changes, available alternatives and costs of false alarms.

The counterfactual must take competitors seriously

A credible 'without-project' world is not an empty desert. Governments adjust procedures, competitors improve products, existing networks detect risks and public budgets evolve. Comparisons should take those alternatives seriously. Value for money may favour a combination of satellite information, local monitoring, physical infrastructure and training rather than the most technologically glamorous single tool.

What to publish when evidence is incomplete

Transparent reporting identifies data provenance, assumptions, missing populations, adverse outcomes, methodological changes and unresolved attribution questions. An independent reviewer should specify whether it examined measurement processes, data quality, selected indicators or causal effects. These are different forms of assurance. The ability to challenge a claim is a strength of responsible impact finance, not a communications failure [R07].