Abstract
The purpose of this article is to introduce the technique and main findings of meta-analysis to the reader, who is unfamiliar with the field and has the usual objections. A meta-analysis is a quantitative survey of a literature reporting estimates of the same parameter. The funnel showing the distribution of the estimates is normally amazingly wide given their t-ratios. Little of the variation can be explained by the quality of the journal (as measured by its impact factor) or by the estimator used. The funnel has often asymmetries consistent with the most likely priors of the researchers, giving a publication bias.
References
Brodeur, A., Lé, M. Sangnier, and Y. Zylberberg (2013). Star wars: The empirics strike back. IZA discussion paper, No. 7268 http://ftp.iza.org/dp7268.pdf.Search in Google Scholar
Callot, L., and M. Paldam (2011). Natural funnel asymmetries. A simulation analysis of the three basic tools of meta-analysis. Research Synthesis Methods 2 (2): 84–102.Search in Google Scholar
Card, D., and A.B. Krueger (1995). Time-series minimum-wage studies: A meta-analysis. American Economic Review 85 (2): 238–243. http://www.jstor.org/stable/2117925?seq=1#page_scan_tab_contentsSearch in Google Scholar
Doucouliagos, C. (H.) (1995). Worker Participation and Productivity in Labor-Managed and Participatory Capitalist Firms: A Meta-Analysis. Industrial and Labor Relations Review 49 (1): 58–75. http://www.jstor.org/stable/2524912?seq=1#page_scan_tab_contents10.1177/001979399504900104Search in Google Scholar
Doucouliagos, H., and M. Paldam (2008). Aid effectiveness on growth. A meta study. European Journal of Political Economy 24 (1): 1–24. http://www.sciencedirect.com/science/article/pii/S0176268007000614Search in Google Scholar
Doucouliagos, H., and M. Paldam (2013). The robust result in meta-analysis of aid effectiveness: A response to Mekasha and Tarp. Journal of Development Studies 49 (4): 584–87. http://www.tandfonline.com/doi/abs/10.1080/00220388.2013.764595#.VSZV_MkrFnU10.1080/00220388.2013.764595Search in Google Scholar
Doucouliagos, H., and M. Paldam (2015). Finally a breakthrough? The recent rise in the size of the estimates of aid effectiveness. For Mak Arvin, ed., Handbook on the Economics of Foreign Aid. Edward Elgar10.4337/9781783474592.00030Search in Google Scholar
Doucouliagos, H., and T.D. Stanley (2012). Are all economic facts greatly exaggerated? Theory competition and selectivity. Journal of Economic Surveys 27 (2): 316–339. http://onlinelibrary.wiley.com/doi/10.1111/j.1467-6419.2011.00706.x/abstractSearch in Google Scholar
Fanelli, D. (2010). ‘‘Positive’’ results increase down the hierarchy of the sciences. Plos ONE 5 (4): 1–10. http://journals.plos.org/plosone/article?id=10.1371/journal.pone.001006810.1371/journal.pone.0010068Search in Google Scholar
Hunter, J.E., and F.L. Schmidt (1990, 2004). Methods of Meta-Analysis: Correcting error and bias in research findings. New York: Sage.Search in Google Scholar
Jarrell, S.B., and T.D. Stanley (1990). A meta-analysis of the union-nonunion wage gap. Industrial and Labor Relations Review 44 (1): 54–67. http://www.jstor.org/stable/2523429?seq=1#page_scan_tab_contents10.1177/001979399004400104Search in Google Scholar
Johnson, R., J. Rolfe, R. Rosenberger, and R. Brouver, eds. (2015). Benefit transfer in environmental and resource values: A handbook for researchers and practitioners. Dordrecht, Springer.Search in Google Scholar
Leamer, E.E. (1983). Let’s take the con out of econometrics. American Economic Review 73 (1): 31–43. http://www.jstor.org/stable/1803924?seq=1#page_scan_tab_contentsSearch in Google Scholar
Mekasha, T.J., and F. Tarp (2013). Aid and growth. What meta-analysis reveals. Journal of Development Studies, 49 (4): 564–583. http://www.tandfonline.com/doi/abs/10.1080/00220388.2012.709621#.VSZZ1skrFnUSearch in Google Scholar
Nelson, J., and P. Kennedy (2010). The use (and abuse) of meta-analysis in environmental and natural resource economics: An assessment. Environmental and Resource Economics 42: 345–377. http://link.springer.com/article/10.1007%2Fs10640-008-9253-5Search in Google Scholar
Paldam, M. (2013). Regression costs fall, mining ratio rise, publication bias looms, and techniques get fancier. Reflections on some trends in empirical macroeconomics. Econ Journal Watch 10 (2): 136–156. http://econjwatch.org/articles/reflections-on-some-trends-in-empiricalmacroeconomicsSearch in Google Scholar
Paldam, M. (2014). How do partly omitted control variables influence the averages used in meta-analysis in economics? Empirical Economics, forthcoming, p.t. at: http://www.martin.paldam.dk/Papers/Meta-method/POCtheory.pdf.Search in Google Scholar
Paldam, M. (2015). Simulating an empirical paper by the rational economist. Aarhus. http://www.martin.paldam.dk/Papers/Meta-method/Simulating-rationality.pdf.Search in Google Scholar
Reed, W.R., R.J.G.M Florax, and J.A. Poot (2015). A Monte Carlo analysis of alternative meta-analysis in the presence of publication bias. Economics: The Open-Access, Open-Assessment E-Journal, forthcoming, p.t. at: http://www.economics-ejournal.org/economics/discussionpapers/2015-910.5018/economics-ejournal.ja.2015-30Search in Google Scholar
Stanley, T.D. (2008). Meta-regression methods for detecting and estimating empirical effect in the presence of publication bias. Oxford Bulletin of Economics and Statistics 70(1): 103–127.Search in Google Scholar
Stanley, T.D., and H. Doucouliagos (2012). Meta-regression analysis in economics and business. Abingdon: Routledge.10.4324/9780203111710Search in Google Scholar
Stanley, T.D., and H. Doucouliagos (2014). Meta-regression approximation to reduce publication selection bias. Research Synthesis Methods 5 (1): 60–78. http://onlinelibrary.wiley.com/doi/10.1002/jrsm.1095/pdf10.1002/jrsm.1095Search in Google Scholar
Summers, L.H. (1991). The scientific illusion in empirical macroeconomics. Scandinavian Journal of Economics 92 (2): 129–148. http://www.jstor.org/stable/3440321?seq=1#page_scan_tab_contents10.2307/3440321Search in Google Scholar
© 2015 Martin Paldam, published by Sciendo
This work is licensed under the Creative Commons Attribution 4.0 International License.
