午夜剧场

I earned my PhD in game theory, a foundational theory of microeconomics, and I still consider theory to be my primary profession. However, for the past few years, I have been writing papers based on experiments. This is because, in the practical application stage of systems designed based on game theory, it has become commonplace to conduct subject experiments in "some form" and present the results of their performance evaluation. However, when asked for the name of the field, I choose to separate the "science" from experimental economics and write it as "economic experiments," meaning subject experiments related to economics. This is because the results are often lacking in evidence to be called scientific knowledge.

Last year and the year before, the clinical trial processes for COVID-19 vaccines were introduced in detail by various media outlets, and I believe many people have gained an understanding of how experimental results are evaluated scientifically or statistically. It is natural to plan experiments so that the attributes of the subjects are as unbiased as possible and to conduct experiments with specifications that can withstand statistical scrutiny (internal validity). However, for many economic experiments, there is no guarantee that the detected results will be detected under the same conditions at another site (external validity). This is because academic journals tend to highly value and accept proposals for new methods that improve the precision of internal validity rather than examining the reproducibility of results.

Such evaluations by leaders in economic experiments are certainly important for claiming that experimental economics is a field independent of theory. Since there are many theories waiting for experimental verification, the secondary justification that conducting experiments in "some form" (guaranteeing internal validity) and showing the results is better than not doing so at all is a reasonable insight, provided that experimental economics is still only a field in the stage of solidifying its scientific foundations. However, even papers that ask whether the same results can be observed under slightly different conditions (robustness) are often rejected outright by many academic journals in economics based on the aforementioned evaluation criteria.

One reason for this current situation is the lack of quantitative evaluation of systems that have begun operation after considering experimental results. It has been less than 20 years since various systems designed based on game theory began to be applied in practice, but operational data for them has been accumulating in countries around the world. If the analysis and sharing of those results progress, the day should be near when economic experiments also begin to focus on guaranteeing external validity.