Estrategias experimentales para el análisis y evaluación de políticas públicas

Luis Miller; Isabel Rodríguez Marín

doi:10.24965/gapp.11133

Authors

Luis Miller Consejo Superior de Investigaciones Científicas IPP-CSIC (España) https://orcid.org/0000-0002-7447-2313
Isabel Rodríguez Marín Consejo Superior de Investigaciones Científicas IPP-CSIC (España) https://orcid.org/0000-0003-2363-3700

DOI:

https://doi.org/10.24965/gapp.11133

Keywords:

Experiments, Evaluation of public policy, Causality, Validity, Experimentation ethics

Abstract

There is increasing concern about the effectiveness of public policies, both at the academic and practical level. Public policy analysis gets more technical every day, and multiple tools are required to establish the causal effect of a policy. In this article, we present an overview of some of the most suitable experimentation-based techniques for this purpose. We examine laboratory and field randomised experiments, on-line experiments and natural experiments. We also address some of the most common caveats of using experiments in public policies, such as validity of results or ethics. Despite possible hassles, the inclusion of experiments in public policy analysis has become the norm in international organisations, and that process is also starting in Spain. Mastering the techniques we present here, as well as understanding their relevance and limitations, is essential for anyone interested in the field of public policy. This article serves as an introduction and guide in that learning. Including experimentation in public policy analysis, we can obtain robust results upon on which build more efficient and effective policies

Downloads

Download data is not yet available.

Author Biographies

Luis Miller, Consejo Superior de Investigaciones Científicas IPP-CSIC (España)

Científico titular del Instituto de Políticas y Bienes Públicos del CSIC. Trabajó como investigador posdoctoral en el Instituto Max Planck de Economía y como adjunto a la dirección e investigador posdoctoral del Centro de Ciencias Sociales Experimentales del Nuffield College de la Universidad de Oxford. Desde 2011 hasta 2018 fue profesor permanente de microeconomía y economía experimental y conductual en la Facultad de Economía y Empresa de la Universidad del País Vasco. Ha sido asesor en la Oficina Nacional de Prospectiva y Estrategia de la Presidencia del Gobierno de España, Director de Gabinete de la Presidencia del CSIC y actualmente preside el Comité ético vinculado a los itinerarios de inclusión social del Ministerio de Inclusión, Seguridad Social y Migraciones. Publica regularmente sobre justicia distributiva, polarización política y métodos experimentales en disciplinas como la economía, ciencia política, sociología, psicología y filosofía

Isabel Rodríguez Marín, Consejo Superior de Investigaciones Científicas IPP-CSIC (España)

Investigadora predoctoral en el Instituto de Políticas y Bienes Públicos del CSIC, con un contrato FPU del Ministerio de Universidades. Doctoranda en el programa Interuniversitario en Economía DEcIDE y colaboradora docente en el Departamento de Economía Aplicada y Estadística de la UNED. Anteriormente ha trabajado en Evaluación en ICEX España y como consultora junior. Sus intereses de investigación giran en torno a la identificación de causalidad entre comportamiento y preferencias políticas, y el contexto socio-económico. En concreto, ha trabajado sobre temas de polarización y movilidad social, y se encuentra investigando sobre normas sociales y discriminación partidista con métodos experimentales

References

Akerlof, G., Oliver, A. y Sunstein, C. (2021). Editorial Introduction: Field experiments and public policy. Behavioural Public Policy, 5(1), número especial. https://www.cambridge.org/core/journals/behavioural-public-policy/issue/8DBD9DA8775493E94495CFCC0796930D

Al-Ubaydli, O., Lee, M., List, J., Mackevicius, C., y Suskind, D. (2021). How can experiments play a greater role in public policy? Twelve proposals from an economic model of scaling. Behavioural Public Policy, 5(1), 2-49. http://doi.org/10.1017/bpp.2020.17

Al-Ubaydli, O. y List, J. A. (2015). On the generalizability of experimental results in economics. En G. Frechette y A. Schotter (eds.), Handbook of Experimental Economic Methodology (pp. 420-462). Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195328325.003.0022

Angrist, J. D. y Pischke, J. S. (2009). Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton University Press. https://doi.org/10.1515/9781400829828

Baldassarri, D. y Abascal, M. (2017). Field experiments across the social sciences. Annual Review of Sociology, 43(1), 41-73. https://doi.org/10.1146/annurev-soc-073014-112445

Barrera, D., Gërxhani, K., Kittel, B., Miller, L. y Wolbring, T. (2023). Experimental Sociology: An Outline of a Scientific Field. Cambridge University Press.

Campbell, D. T., y Stanley, J. C. (1963). Experimental and Quasi-Experimental Designs for Research. Rand McNally & Company. https://www.jameslindlibrary.org/campbell-dt-stanley-jc-1963/

Caparrós, A., Cerdá, E., Ovando, P. y Campos, P. (2007, 6 de abril). Carbon Sequestration with Reforestations and Biodiversity-Scenic Values [FEEM Working Paper 28]. Fondazione Eni Enrico Mattei. https://services.bepress.com/feem/paper31/

Cook, T. D., Y Campbell, D. T. (1979). Quasi-Experimentation: Design & Analysis Issues for Field Settings. Houghton Mifflin.

Deaton, A. y Cartwright, N. (2018). Understanding and misunderstanding randomized controlled trials. Social Science & Medicine, (210), 2-21. https://doi.org/10.1016/j.socscimed.2017.12.005

Dinas, E., Matakos, K., Xefteris, D., y Hangartner, D. (2019). Waking Up the Golden Dawn: does exposure to the refugee crisis increase support for extreme-right parties? Political analysis, 27(2), 244-254. https://cadmus.eui.eu/handle/1814/61887

Dunning, T. (2012). Natural Experiments in the Social Sciences. A Design-based Approach. Cambridge University Press. https://doi.org/10.1017/cbo9781139084444

Falk, A. y Heckman, J. J. (2009). Lab experiments are a major source of knowledge in the social sciences. Science, 326(5952), 535-538. https://doi.org/10.1126/science.1168244

Gereke, J. y Gërxhani, K. (2019). Experimental Economics and Experimental Sociology. En J. H. Hamilton, A. Dixit, S. Edwards y K. Judd (eds.). Oxford Research Encyclopedia, Economics and Finance. Oxford University Press. https://doi.org/10.1093/acrefore/9780190625979.013.462

Gerber, A. S., y Green, D. P. (2012). Field experiments. Design, analysis, and interpretation. W. W. Norton.

Gertler, P. (2004). Do Conditional Cash Transfers Improve Child Health? Evidence from PROGRESA’s Control Randomized Experiment. American Economic Review, 94(2), 336-341. https://www.aeaweb.org/articles?id=10.1257/0002828041302109

Gertler, P., Martínez, S., Premand, P., Rawlings, L. B. y Vermeersch, C. M. J. (2016). La Evaluación de Impacto en la Práctica (2.ª Edición). Inter-American Development Bank – World Bank. https://openknowledge.worldbank.org/handle/10986/25030

Gërxhani, K. (2020). Status Ranking and Gender Inequality: A Cross-Country Experimental Comparison. Research in Social Stratification and Mobility, (65), artículo 100474. https://doi.org/10.1016/j.rssm.2020.100474

Gërxhani, K., Brandts, J. y Schram, A. (2021, septiembre). Competition and gender inequality: A comprehensive analysis of effects and mechanisms [BSE Working Paper 1292]. Barcelona School of Economics. https://bse.eu/research/working-papers/competition-and-gender-inequality-comprehensive-analysis-effects-and

Gërxhani, K. & Miller, L. (2022). Experimental Sociology. En K. Gërxhani, N. D. de Graaf y W. Raub (eds.), Handbook of Sociological Science. Contributions to Rigorous Sociology (pp. 309-323), Edward Elgar Publishing.

Glennerster, R. y Takavarasha, K. (2013). Running Randomized Evaluations: A Practical Guide. Princeton University Press.

Harrison, G. W. y List, J. A. (2004). Field experiments. Journal of Economic Literature, 42(4), 1009-1055. https://doi.org/10.1257/0022051043004577

Hey, J. D. (1998). Experimental economics and deception: A comment. Journal of Economic Psychology, 19(3), 397-401. https://doi.org/10.1016/s0167-4870(98)00013-0

Jiménez-Buedo, M. y Miller, L. (2010). Why a Trade-Off? The Relationship Between the Internal and External Validity of Experiments. THEORIA, An International Journal for Theory, History and Foundations of Science, 25(3), 301-321. https://ojs.ehu.eus/index.php/THEORIA/article/view/779

John, P. (2017). Field Experiments in Political Science and Public Policy. Practical Lessons in Design and Delivery. Routledge. https://doi.org/10.4324/9781315773025

Miller, L. (2006). Experimentos de orientación teórica: una discusión metodológica, Empiria (12), 89-110.

Morton, R. B. y Williams, K. C. (2010). Experimental Political Science and the Study of Causality. From Nature to the Lab. Cambridge University Press. https://doi.org/10.1017/cbo9780511762888

Mutz, D. C. (2011). Population-Based survey experiments. Princeton University Press. https://doi.org/10.23943/princeton/9780691144511.001.0001

Nakamura, E., Sigurdsson, J. y Steinsson, J. (2022) The Gift of Moving: Intergenerational Consequences of a Mobility Shock. The Review of Economic Studies, 89(3), 1557-1592. https://doi.org/10.1093/restud/rdab062

Oakley A. (1998). Experimentation and social interventions: a forgotten but important history. British Medical Journal, 317(7167), 1239-1242. https://doi.org/10.1136/bmj.317.7167.1239

Parker, S. W. y Teruel, G. M. (2005). Randomization and Social Program Evaluation: The Case of Progresa. The ANNALS of the American Academy of Political and Social Science, 599(1), 199-219. https://doi.org/10.1177/0002716205274515

Rubin, D. (1974). Estimating Causal Effects of Treatments in Randomized and Non-Randomized Studies. Journal of Educational Psychology 66(5), 688-701. https://doi.org/10.1037/h0037350

Salganik, M. J. (2019). Bit by Bit: Social research in the Digital Age. Princeton University Press.

Schram, A., Brandts, J. y Gërxhani, K. (2019). Status ranking: a hidden channel to gender inequality under competition. Experimental Economics, (22), 396-418. https://doi.org/10.1007/s10683-018-9563-6

Shadish, W. R., Cook, T. D. y Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton, Mifflin and Company.

Webster, M. y Sell, J. (2014). Laboratory Experiments in the Social Sciences. Elsevier. https://doi.org/10.1016/C2011-0-07562-2

Weiss, C. H. y Birckmayer, J. (2009) Social Experimentation for Public Policy. En R. Goodin, M. Moran y M. Rein (eds.), The Oxford Handbook of Public Policy (pp. 806-830). Oxford University Press. https://doi.org/10.1093/oxfordhb/9780199548453.003.0039