A recent study by the Federal Reserve has shed light on the lack of replicability in economic research, showing that many published papers cannot be reproduced. The study, conducted by economists Andrew Chang and Phillip Li, attempted to replicate 67 papers from 13 well-regarded economics journals and found that they were only able to replicate 49% of them with assistance from the original authors. This raises questions about the validity of the research that policymakers and the public rely on for making important decisions.
One of the challenges faced by Chang and Li was obtaining data and code from the original papers, as some journals required it while others did not. This led to difficulties in replicating the results, as they were only able to access data and code for a portion of the papers. The study also found that some papers used confidential data that was unavailable, further complicating the replication process.
The implications of this lack of replicability are significant, especially when considering the influence that economic research has on policy decisions. For example, during a recent period of inflation, many economists pointed to the Phillips Curve as a justification for raising unemployment to combat rising prices, despite its unreliable nature. If a significant portion of economic research cannot be replicated, it calls into question the validity of these types of conclusions.
The study was prompted by a recent article in The Economist that highlighted the inaccuracies in the popular book “Freakonomics,” which made controversial claims based on flawed data analysis. The book’s assertions about the relationship between abortion legalization and crime rates were later found to be incorrect due to a coding error and the use of inappropriate data. This case serves as a cautionary tale about the dangers of relying on economic research without proper scrutiny.
The Federal Reserve study suggests that the field of economics still has a long way to go in terms of ensuring the validity and replicability of its research. While the questions and challenges in economics remain as interesting as ever, it is clear that there is much work to be done in developing reliable tools for investigating economic phenomena. Moving forward, it is essential for researchers, policymakers, and the public to approach economic research with a critical eye and a healthy dose of skepticism.
