Use of Machine Learning in Economic Research: What the Literature Tells Us

A selective review of 20 outstanding articles published in leading economic journals over the last four decades.

Machine learning topics started to appear in economic literature on a larger scale in the 1980s when the main concepts such as backpropagation, recurrent neural networks (RNNs) and restricted Boltzmann machines (RBM) were discovered, and topics like computer vision attracted a lot of attention. The increasing trend is visible in the frequency of articles dealing with machine learning published in four leading economic journals over the last couple of decades (see the chart below).

In this article, I share the findings from the literature on machine learning applications in economics and briefly review 20 articles I find most influential in this area. It is a subjective selection of papers from A economic journals (based on the article influence score) that might be useful for applied researchers planning to dive deeper into this exciting field or for anyone interested in AI spillovers into other areas.

I identify four main areas where ML methods are used in economic literature:

• Forecasting and predictions
• Sentiment analysis and Natural Language Processing (NLP)
• Image processing and computer vision
• Process automation and optimization

Finally, I refer to the leading economist’s vision for the future and include some remarks on where the whole field of AI in economics is going.

Early attempts to incorporate AI into economics

One of the first articles on this topic is a 1984 American Economic Review (AER) article by Cohen and Axelrod that shows a model of updating beliefs to new experience that relies on a rather archaic AI program for playing checkers. The authors show that the model performs better than the other state-of-the-art models.

In an exciting piece of research, Arthur (AER, 1991) designs a parametrized learning automaton, an algorithm to “reproduce” human behavior and calibrates it against human subjects. The paper concludes that we can design artificial learning agents and calibrate their “rationality” to replicate human behavior.

Complex adaptive AI systems and the economic agents’ behavior are studied by Holland and Miller (AER, 1991). These days, a wide range of computer-based adaptive algorithms existed for exploring such systems, including classifier systems, genetic algorithms, neural networks, and reinforcement learning mechanisms.

Process automation

An ideal use case of ML in any field is optimization, and processes automation, and in this sense saving labor costs and (or) increasing productivity. For example, Fernald and Jones (AER, 2014) suggest that artificial intelligence will allow machines to replace workers, which to some extent could lead to higher growth in the future. New research technologies might enable computers and robots to replace labor, dramatically impacting production and economic growth.

Forecasting and predictions

Supervised ML has been used in many applications for forecasting where it reaches magnificent performance. Hyperparameter tuning, train-validation-test split, and improved optimization algorithms such as ADAM contribute to better performance of ML compared to some other statistical methods.

Kleinberg et al. (AER, 2015) and Kleinberg et al. (QJE, 2017) highlight how improved prediction using machine learning techniques can have large policy impacts. Many problems can be tackled in this way. For instance, in the criminal justice system, judges have to decide whether to detain or release arrestees. A decision that depends on a prediction about the arrestee’s probability of committing a crime could be supported by ML.

Other examples include: (1) in education, predicting which teacher will have the greatest value-added; (2) in labor market policy, predicting unemployment spell length to help workers decide on savings rates and job search strategies; (3) in regulation, targeting health inspections; (4) in social policy, predicting highest risk youth for targeting interventions; and (5) in the finance sector, lenders identifying the underlying credit-worthiness of potential borrowers (Kleinberg at al., 2015).

On a similar note, Chalfin et al. (AER, 2015) demonstrate the social welfare gains resulting from using ML to improve predictions of worker productivity. They illustrate the value of this approach in two essential applications — police hiring decisions and teacher tenure decisions.

Bajari et al. (AER, 2015) review and apply several popular ML methods to demand estimation. They show that these methods can produce superior predictive accuracy compared to a standard linear regression or logit model.

There are many other examples where ML is used to predict variables that are important for policy decisions. In international economics, Amat et al. (JIMF, 2018) use sequential ridge regression to forecast exchange rates for major international currencies. It is a practical example of ML implementation for policymaking, as they conclude that their ML model leads to improved forecast and lower root mean square error (RMSE).

Sentiment analysis and NLP

The increasing availability of labeled training datasets stimulated researchers to use sentiment analysis on various problems in economics. The sentiment datasets with labels for each part of the text (for example, 1 = positive, 0 = negative; 1 : 25 being 1 the most negative and 25 the most positive, etc.) help researchers make classifiers that can be used in their own datasets.

With big NLP and language modeling advances in the 2010s, a couple of nice papers appeared in top journals. Some review articles include Algaba et al. (JES, 2020), overviewing semantic analytics methods that use textual, audio, and visual data in economic science. Gentzkow et al. (JEL, 2019) focus purely on text data and review appropriate statistical methods and various applications to economic problems.

In monetary economics, Hansen et al. (QJE, 2017) use computational linguistics to discover central bankers’ communication patterns. Quantification of text data allows to measure the impact of central bankers’ statements and ease the construction of rich communication measures. Hansen and McMahon (JIE, 2016) investigate the same topic with a similar text-based approach.

NLP techniques might help to improve the accuracy of standard statistical methods. For example, in corporate finance, Goberg and Maksimovic (RFS, 2014) improve the measurement of financial constraints (i.e., the ability of firms to access finance). The text-based measure outperforms others used in the literature in predicting investment cuts following adverse economic shocks.

Business economics applications of NLP include Bandiera (JPE, 2020) constructing a CEO behavior index employing a widely-used NLP algorithm (Latent Dirichlet Allocation — LDA). Apart from good performance, it also allows reducing the high dimensionality of the data.

Image processing and computer vision

Image processing and graphical data processing articles commonly use various Convolutional Neural Nets (CNNs) and frequently use big data. Many datasets are already freely available, and researchers can use pre-processed data or pre-trained models for their applications (for completeness, this process is called transfer learning).

Donaldson and Storeygard (JEP, 2016) review the literature of articles using satellite data (remotely sensed data from the orbit). Small satellites are flying over the Earth at some distance and collect photographic images and other data, which can be used for studying processes on the Earth. For example, Henderson et al. (AER, 2012) use satellite data to measure economic activity (GDP growth) at the sub- and supranational regions, which is impossible with the current measures.

Naik et al. (PNAS, 2016) use 360-degree panorama images of streetscapes to predict the determinants of physical urban change (physical appearance of streets) in five US cities. Naik et al. (AER, 2016) focus on an empirical connection between the physical appearance of a town and the behavior and health of its inhabitants. They use a Support Vector Machine to quantify urban appearance from street-level images.

Where are we going?

What are the directions for future research where the research of AI in economics is going? Let’s get inspired by Susan Athey (Stanford Graduate School of Business), who predicts inventions coming in the near future.

I include an exact reference from Athey (2019):

1. Adoption of off-the-shelf ML methods for their intended tasks (prediction, classification, and clustering, e.g., for textual analysis).
2. Extensions and modifications of prediction methods to account for considerations such as fairness, manipulability, and interpretability.
3. Development of new econometric methods based on machine learning designed to solve traditional social science estimation tasks.
4. Incremental progress to identification and estimation strategies for causal effects that exploit modern data settings including large-panel datasets and environments with many small experiments.
5. Increased emphasis on model robustness and other supplementary analysis to assess credibility of studies.
6. Adoption of new methods by empiricists at large scale.
7. Revival and new lines of research in productivity and measurement.
8. New methods for the design and analysis of large administrative data, including merging these sources and privacy-preserving methods.
9. Increase in interdisciplinary research.
10. Changes in organization, dissemination, and funding of economic research.
11. Economist as engineer engages with firms, government to design, and implement policies in digital environment.
12. Design and implementation of digital experimentation, both one-time and as an ongoing process, including multiarmed bandit experimentation algorithms, in collaboration with firms and government.
13. Research on developing high-quality metrics that can be measured quickly, in order to facilitate rapid incremental innovation and experimentation.
14. Increased use of data analysis in all levels of economics teaching; increase in interdisciplinary data science programs.
15. Research on the impact of AI and ML on the economy.

Conclusions

I have included 20 A-class articles that show numerous applications of ML methods in economics. In the coming future, we can expect rapid growth of published articles in economic journals and the implementation of new techniques that the AI community is developing. At the moment, ML is still an applied statistical method, but with its unique statistical methodology, it outperforms standard statistical methods in many cases. Especially in forecasting, it dominates the field in terms of the accuracy of predictions.

The last paper I would like to mention is Mullainathan and Spiess (JEP, 2017) that offers a great introduction to the theme.

Did you like the article? You can invite me for coffee and support my writing. You can also subscribe to my email list to get notified about my new articles. Thanks!

References

[1] Michael D. Cohen and Robert Axelrod, 1984: Coping with Complexity: The Adaptive Value of Changing Utility, The American Economic Review, Vol. 74, No. 1 (Mar., 1984), pp. 30-42.

[2 ] W. Brian Arthur, 1991: Designing Economic Agents that Act like Human Agents: A Behavioral Approach to Bounded Rationality, The American Economic Review, Vol. 81, No.2, Papers and Proceedings. (May, 1991), pp. 353–359.

[3] John H. Holland and John H. Miller, 1991: Artificial Adaptive Agents in Economic Theory, The American Economic Review, Vol. 81, No. 2, Papers and Proceedings. (May, 1991), pp. 365–370.

[4] John G. Fernald and Charles I. Jones, 2014: The Future of US Economic Growth, The American Economic Review, Vol. 104, No. 5, Papers and Proceedings. (May 2014), pp. 44–49.

[5] Jon Kleinberg, Jens Ludwig, Sendhil Mullainathan, and Ziad Obermeyer, 2015: Prediction Policy Problems, The American Economic Review, Vol. 105, No. 5. , Papers and Proceedings, pp. 491–495.

[6] Jon Kleinberg, Himabindu Lakkaraju, Jure Leskovec, Jens Ludwig, Sendhil Mullainathan, 2017: Human Decisions and Machine Predictions, The Quarterly Journal of Economics, Vol. 133, No.1, (February 2018), pp. 237–293.

[7] Aaron Chalfin, Oren Danieli, Andrew Hillis, Zubin Jelveh, Michael Luca, Jens Ludwig, And Sendhil Mullainathan, 2015: Productivity and Selection of Human Capital with Machine Learning, The American Economic Review, Vol. 106, No. 5, (May 2016), pp. 124–27.

[8] Patrick Bajari, Denis Nekipelov, Stephen P. Ryan and Miaoyu Yang, 2015: Machine Learning Methods for Demand Estimation, The American Economic Review, Vol. 105, No. 5, Papers and Proceedings (May 2015), pp. 481–485.

[9] Christophe Amat, Tomasz Michalski, Gilles Stoltz, 2018: Fundamentals and exchange rate forecastability with simple machine learning methods, Journal of International Money and Finance, Vol. 88, November 2018, pp. 1–24.

[10] Andres Algaba, David Ardia, Keven Bluteau, Samuel Borms, Kris Boudt, 2020: Econometrics Meets Sentiment: An Overview Of Methodology And Applications, Journal of Economic Surveys, Vol. 34, Issue 3, July 2020, pp. 512–547.

[11] Matthew Gentzkow, Bryan Kelly, and Matt Taddy, 2019: Text as Data, Journal of Economic Literature, Vol., 57, No. 3, pp: 535–574.

[12] Stephen Hansen Michael Mcmahon Andrea Prat, 2018: Transparency And Deliberation Within The Fomc: A Computational Linguistics Approach, The Quarterly Journal of Economics, Vol. 133, No. 2, May 2018, pp. 801–870.

[13] Gerard Hoberg, Vojislav Maksimovic, 2017: Redefining Financial Constraints: A Text-Based Analysis, The Review of Financial Studies, Vol. 28, No. 5, May 2015, pp. 1312–1352.

[14] Oriana Bandiera, Andrea Prat, Stephen Hansen, Raffaella Sadun, 2020: Journal of Political Economy, Vol. 128, No., 4, April 2020.

[15] Dave Donaldson and Adam Storeygard, 2016: The View from Above: Applications of Satellite Data in Economics, Journal of Economic Perspectives, Vol. 30, No. 4, pp. 171–198.

[16] J. Vernon Henderson, Adam Storeygard, David N. Weil, 2012: Measuring Economic Growth from Outer Space, The American Economic Review, Vol. 102, No. 2, pp. 994–1028.

[17] Nikhil Naik, Scott Duke Kominers, Ramesh Raskar, Edward L. Glaeser, and César A. Hidalgo, 2017: Computer vision uncovers predictors of physical urban change, Proceedings of the National Academy of Sciences of the United States of America, July 18, Vol. 114, No. 29, pp. 7571–7576.

[18] Nikhil Naik, Ramesh Raskar, César A. Hidalgo, 2016: Cities Are Physical Too: Using Computer Vision to Measure the Quality and Impact of Urban Appearance, The American Economic Review, Vol. 106, No. 5, May 2016, pp. 128–32.

[19] Susan Athey, 2019: The Impact of Machine Learning on Economics, In: The Economics of Artificial Intelligence: An Agenda, University of Chicago Press, May 2019, pp. 507–547.

[20] Sendhil Mullainathan and Jann Spiess, 2017: Machine Learning: An Applied Econometric Approach, Journal of Economic Perspectives, Vol. 31, No. 2, Spring 2017 , pp. 87–106.