Measuring the Unmeasured: Combining Technology and Behavioral Insights to Improve Measurement of Business Outcomes

Working Paper
Published on 1 April 2019

Abstract

Business survey outcomes for micro and small firms are notoriously noisy, with multiple sources of measurement and recall error. Anderson et al. (2019) introduce a new survey methodology that combines automatic consistency checks of electronic data collection with triangulation and dynamic adjustment to arrive at more precise estimates of business performance. The methodology uses insights from behavioral science to lower the cognitive cost of initial recall and establishes salient and relevant anchors to allow for dynamic triangulation and adjustment toward a final estimate. The validity of this method is field tested against traditional performance measures as well as administrative data across three emerging markets: Ghana, Rwanda, and Uganda. The results show significant upward adjustment from traditional measures for both sales and profits, a lower coefficient of variation in the cross-section, and higher autocorrelation in panel data. Comparisons with administrative data further confirm a higher correlation and closer magnitude relative to  traditional  measures.  This  research  reconciles  recommendations for increased attention to survey design with a method to leverage electronic survey technology beyond consistency checks.

Authors

Stephen Anderson

Stanford University

Christy Lazicky

IDinsight

Bilal Zia

World Bank