Iranian Journal of Information Processing and Management

Iranian Journal of Information Processing and Management

Challenges and Complexities in Leveraging Data for Evidence-Based Policy making A Scoping Review

Document Type : Original Article

Authors
Faculty of Management.University of Tehran. Tehran. IRAN
Abstract
Evidence-based policymaking stands at the forefront of contemporary governance, where data and evidence have emerged as indispensable allies in shaping effective and informed decisions. This article embarks on a comprehensive exploration of the challenges and critical issues encountered when data assumes the role of evidence in policy formulation. The foundation of this investigation is laid within the rich tapestry of evidence-based policy-making literature. We delve into the scholarly discourse, tracing the evolution of policy formulation from the realm of intuition to one guided by empirical insights. As we traverse this intellectual landscape, the pivotal role of data as a catalyst for this transformation becomes apparent. Delving deeper, we dissect the nuances of data and the emergence of big data. Once regarded as mere numbers, data now embodies the currency of the information age. Its volume, velocity, and variety characterize it, rendering it a potent tool for evidence generation and policy formulation. As we explore its features, we unveil the potential of data to unlock unprecedented insights and inform governance with empirical precision. To undertake a systematic exploration of the challenges faced in using data as evidence in policy-making, we employ a rigorous scoping review methodology. Through meticulous screening, we identify and analyze 36 exemplary articles that offer invaluable insights into the multifaceted landscape of data-driven governance. These articles provide a comprehensive panorama of the challenges, grouped into three distinct clusters: technical challenges emanating from data complexities, legal and privacy dilemmas entwined in governance, and the formidable issues policymakers face. Our discussion unravels the intricate web of challenges, from data quality and integration to confidentiality, ethics, and governance issues. We explore the nuances of data access, the battle against bias, and the intricacies of data volume and complexity. Simultaneously, we delve into the legal labyrinth of data ownership, security, sharing, and compliance. The challenges policymakers face in fostering data-driven cultures, navigating resource constraints, and communicating data-driven insights are brought to the fore. In conclusion, our exploration illuminates the multifaceted challenges and critical issues underpinning data utilization as evidence in evidence-based policy-making. This research underscores the transformative power of data in governance and emphasizes the challenges and pressing issues of using data as evidence in policymaking.
Keywords
Subjects

 

References

Abraham, R., Schneider, J., & Vom Brocke, J. (2019). Data governance: A conceptual framework, structured review, and research agenda. International journal of information management, 49, 424-438. DOI: 10.1016/j.ijinfomgt.2019.07.008
Agarwal, P. K. (2018). Public administration challenges in the world of AI and bots. Public Administration Review, 78(6), 917-921. DOI:10.1111/puar.12979
Ali, H. S. (2021). Analytical Level of Discourse Analysis. Social Science and Humanities, 1(1), 28-37.
Banks, G. (2010). Evidence-based policy making: What is it? How do we get it?. In WORLD SCIENTIFIC REFERENCE ON ASIA-PACIFIC TRADE POLICIES: 2: Agricultural and Manufacturing Protection in Australia (pp. 719-736). DOI: 10.1142/9789813274754_0025
Melnyk, B., Fineout-Overholt, E., Stone, P., & Ackerman, M. (2005). Evidence-based practice. Nursing and healthcare: A guide to best practice. Philadelphia: Lippincott Williams & Wilkins.
Botterill, L. C., & Hindmoor, A. (2012). Turtles all the way down: Bounded rationality in an evidence-based age. Policy Studies, 33(5), 367-379. DOI:10.1080/01442872.2011.626315.
Bradley, C. J., Penberthy, L., Devers, K. J., & Holden, D. J. (2010). Health services research and data linkages: issues, methods, and directions for the future. Health services research, 45(5p2), 1468-1488. DOI:10.1111/j.1475-6773.2010.01142.x
Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C., & Hung Byers, A. (2011). Big data: The next frontier for innovation, competition, and productivity.
Bryant, R., Katz, R. H., & Lazowska, E. D. (2008). Big-data computing: creating revolutionary breakthroughs in commerce, science and society.
Cairney, P. (2016). The politics of evidence-based policy making. Springer.
Cairney, P., & Kwiatkowski, R. (2017). How to communicate effectively with policymakers: combine insights from psychology and policy studies. Palgrave Communications, 3(1), 1-8. DOI: 10.1057/s41599-017-0046-8
Carmi, E., & Yates, S. J. (2020). What do digital inclusion and data literacy mean today?. Internet Policy Review, 9(2). DOI: 10.14763/2020.2.1474
Chambers, A. (2002). The chambers dictionary. Allied Publishers.
Cairney, P., & Cairney, P. (2016). The role of evidence in theories of the policy process. The politics of evidence-based policy making, 13-50.
Chang, V. (2021). An ethical framework for big data and smart cities. Technological Forecasting and Social Change, 165, 120559. DOI: 10.1016/j.techfore.2020.120559
Chen, J., Chen, Y., Du, X., Li, C., Lu, J., Zhao, S., & Zhou, X. (2013). Big data challenge: a data management perspective. Frontiers of computer Science, 7, 157-164. DOI: 10.1007/s11704-013-3903-7
Clark, W. R., & Golder, M. (2015). Big data, causal inference, and formal theory: Contradictory trends in political science?: Introduction. PS: Political Science & Politics, 48(1), 65-70. DOI: DOI: 10.1017/S1049096514001759
Davies, P. (2012). What is Evidence-Based Education?. Education Matters: 60 years of the British Journal of Educational Studies. Doi: 10.4324/9780203123829-14
De Marchi, G., Lucertini, G., & Tsoukiàs, A. (2016). From evidence-based policy making to policy analytics. Annals of operations research, 236(1), 15-38. DOI: 10.1007/s10479-014-1578-6
Denning, P. J. (1990). The science of computing: Saving all the bits. American Scientist, 78(5), 402-405.
Desouza, K. C., & Jacob, B. (2017). Big data in the public sector: Lessons for practitioners and scholars. Administration & society, 49(7), 1043-1064. DOI: 10.1177/0095399714555751
Dommett, K. (2019). Data-driven political campaigns in practice: understanding and regulating diverse data-driven campaigns. Internet Policy Review, 8(4).
Dowie, J. (1996). Evidence based medicine. Needs to be within framework of decision making based on decision analysis. BMJ: British Medical Journal, 313(7050), 170. DOI: 10.1136/bmj.313.7050.170a
Esty, D., & Rushing, R. (2007). The promise of data-driven policymaking. Issues in Science and Technology, 23(4), 67-72.
Ferrandino, J. (2014). The enemy of teaching evidence-based policy: The Powell-Bush doctrine of public affairs. Journal of Public Affairs Education, 20(1), 73-89. DOI: 10.1080/15236803.2014.12001772
Gambrill, E. (2006). Evidence-based practice and policy: Choices ahead. Research on social work practice, 16(3), 338-357. DOI:10.1177/1049731505284205
George, G., Haas, M. R., & Pentland, A. (2014). Big data and management. Academy of management Journal, 57(2), 321-326. https://doi.org/10.5465/amj.2014.4002
Geyer, R., & Harrison, N. E. (2021). From Order to Complexity in Policy and Governance. In Governing Complexity in the 21st Century (pp. 33-46). Routledge.
Gray, J.A.M. (1997). Evidence-Based Healthcare; Churchill Livingstone: New York, NY, USA.
Gutierrez-Bucheli, L., Reid, A., & Kidman, G. (2022). Scoping reviews: Their development and application in environmental and sustainability education research. Environmental Education Research, 28(5), 645-673. DOI: 10.1080/13504622.2022.2047896
Habibzadeh, H., Nussbaum, B. H., Anjomshoa, F., Kantarci, B., & Soyata, T. (2019). A survey on cybersecurity, data privacy, and policy issues in cyber-physical system deployments in smart cities. Sustainable Cities and Society, 50, 101660. DOI: 10.1016/j.scs.2019.101660
Head, B. W. (2010). Reconsidering evidence-based policy: Key issues and challenges. Policy and society, 29(2), 77-94. DOI: 10.1016/j.polsoc.2010.03.001
Head, B. W. (2016). Toward more “evidence‐informed” policy making?. Public administration review, 76(3), 472-484. DOI: 10.1111/puar.12475
Hong, S., Hyoung Kim, S., Kim, Y., & Park, J. (2019). Big Data and government: Evidence of the role of Big Data for smart cities. Big data & society, 6(1), 2053951719842543. DOI: 10.1177/2053951719842543
Hummel, P., Braun, M., & Dabrock, P. (2021). Own data? Ethical reflections on data ownership. Philosophy & Technology, 34(3), 545-572.  DOI: 10.1007/s13347-020-00404-9
Janssen, M., & Kuk, G. (2016). The challenges and limits of big data algorithms in technocratic governance. Government Information Quarterly, 33(3), 371-377. DOI: 10.1016/j.giq.2016.08.011
Janssen, M., & van den Hoven, J. (2015). Big and Open Linked Data (BOLD) in government: A challenge to transparency and privacy?. Government Information Quarterly, 32(4), 363-368. DOI: 10.1016/j.giq.2015.11.007
Janssen, M., Brous, P., Estevez, E., Barbosa, L. S., & Janowski, T. (2020). Data governance: Organizing data for trustworthy Artificial Intelligence. Government Information Quarterly, 37(3), 101493. DOI: 10.1016/j.giq.2020.101493
Janssen, M., Brous, P., Estevez, E., Barbosa, L. S., & Janowski, T. (2020). Data governance: Organizing data for trustworthy Artificial Intelligence. Government Information Quarterly, 37(3), 101493. DOI: 10.1016/j.giq.2020.101493
Jennings Jr, E. T., & Hall, J. L. (2012). Evidence-based practice and the use of information in state agency decision making. Journal of Public Administration Research and Theory, 22(2), 245-266.DOI: 10.1093/jopart/mur040
Joseph Rowntree Foundation. (2000). Linking research and practice. York: Joseph Rowntree Foundation.http://www.jrf.org.uk
Jost, J. T., Federico, C. M., & Napier, J. L. (2009). Political ideology: Its structure, functions, and elective affinities. Annual review of psychology, 60, 307-337. DOI: 10.1146/annurev.psych.60.110707.163600
Kankanhalli, A., Charalabidis, Y., & Mellouli, S. (2019). IoT and AI for smart government: A research agenda. Government Information Quarterly, 36(2), 304-309. DOI: 10.1016/j.giq.2019.02.003
Kempeneer, S. (2021). A big data state of mind: Epistemological challenges to accountability and transparency in data-driven regulation. Government Information Quarterly, 38(3), 101578. DOI: 10.1016/j.giq.2021.101578
 
Kim, E. S., Choi, Y., & Byun, J. (2019). Big data analytics in government: Improving decision making for R&D investment in Korean SMEs. Sustainability, 12(1), 202. DOI: 10.3390/su12010202
Kimani, K., Oduol, V., & Langat, K. (2019). Cyber security challenges for IoT-based smart grid networks. International journal of critical infrastructure protection, 25, 36-49. DOI: 10.1016/j.ijcip.2019.01.001
Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabási, A. L., Brewer, D., ... & Van Alstyne, M. (2009). Computational social science. Science, 323(5915), 721-723. DOI: 10.1126/science.1167742
Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014). The parable of Google Flu: traps in big data analysis. science, 343(6176), 1203-1205.DOI: 10.1126/science.1248506
Löfgren, K., & Webster, C. W. R. (2020). The value of Big Data in government: The case of ‘smart cities’. Big Data & Society, 7(1), 2053951720912775.DOI: 10.1177/2053951720912775
Longo, J., & Dobell, A. R. (2018). The limits of policy analytics: Early examples and the emerging boundary of possibilities.DOI:10.17645/pag.v6i4.1561
Malomo, Fola, and Vania Sena. "Data intelligence for local government? Assessing the benefits and barriers to using big data in the public sector." Policy & Internet 9, no. 1 (2017): 7-27. DOI: 10.1002/poi3.141
Matheus, R., Janssen, M., & Maheshwari, D. (2020). Data science empowering the public: Data-driven dashboards for transparent and accountable decision-making in smart cities. Government Information Quarterly, 37(3), 101284. https://doi.org/10.1016/j.giq.2018.01.006
Mergel, I., Rethemeyer, R. K., & Isett, K. (2016). Big data in public affairs. Public Administration Review, 76(6), 928-937.DOI: 10.1111/puar.12625
Mulgan, G. (2003). Government, knowledge and the business of policy-making. Canberra Bulletin of Public Administration, (108), 1-5.
Namdarian, L. (2016). Evidence-based policy-making and the role of statistics and information. Iranian journal of information processing and management, 31 (85), 601-629.
Namdarian, L. (2019). Analyzing appropriate features of data for using in policy-making. Politics Quartrly, 49 (2), 515-534.
Perry, B., & Uuk, R. (2019). AI governance and the policymaking process: key considerations for reducing AI risk. Big data and cognitive computing, 3(2), 26. https://doi.org/10.3390/bdcc3020026
Pirog, M. A. (2014). Data will drive innovation in public policy and management research in the next decade. Journal of Policy Analysis and Management, 537-543.
https://www.jstor.org/stable/24033344
Rantos, K., Spyros, A., Papanikolaou, A., Kritsas, A., Ilioudis, C., & Katos, V. (2020). Interoperability challenges in the cybersecurity information sharing ecosystem. Computers, 9(1), 18. https://doi.org/10.3390/computers9010018
Rawat, D. B., Doku, R., & Garuba, M. (2019). Cybersecurity in big data era: From securing big data to data-driven security. IEEE Transactions on Services Computing, 14(6), 2055-2072. DOI: 10.1109/TSC.2019.2907247
Rodríguez, P., Palomino, N., & Mondaca, J. (2017). Using big data and its analytical techniques for public policy design and implementation in Latin America and the Caribbean. DOI: 10.18235/0000694
Sander, I. (2020). What is critical big data literacy and how can it be implemented?. Internet Policy Review, 9(2). DOI:10.14763/2020.2.1479
Seppi, K., Ray Chaudhuri, K., Coelho, M., Fox, S. H., Katzenschlager, R., Perez Lloret, S., ... & Djamshidian‐Tehrani, A. (2019). Update on treatments for nonmotor symptoms of Parkinson's disease—an evidence‐based medicine review. Movement Disorders, 34(2), 180-198. DOI:10.1002/mds.27602
Shaxson, L. (2005). Is your evidence robust enough? Questions for policy makers and practitioners. Evidence & Policy, 1(1), 101-111. DOI:10.1332/1744264052703177
Spiel, C., & Strohmeier, D. (2012). Evidence-based practice and policy: When researchers, policy makers, and practitioners learn how to work together. European Journal of Developmental Psychology, 9(1), 150-162. https://doi.org/10.1080/17405629.2011.616776
Starke, C., & Lünich, M. (2020). Artificial intelligence for political decision-making in the European Union: Effects on citizens’ perceptions of input, throughput, and output legitimacy. Data & Policy, 2, e16. DOI: 10.1017/dap.2020.19
Stone, D., & Denham, A. (Eds.). (2004). Think tank traditions: Policy research and the politics of ideas. Manchester: Manchester University Press.
Sutcliffe, S. (2005). Evidence-Based Policymaking: What is it? How does it work? What relevance for developing countries?.
Tan, S. Y., Taeihagh, A., & Pande, D. (2023). Data sharing in disruptive technologies: lessons from adoption of autonomous systems in Singapore. Policy Design and Practice, 6(1), 57-78. DOI: 10.1080/25741292.2022.2162247
Tomar, L., Guicheney, W., Kyarisiima, H., & Zimani, T. (2016). Big Data in the public sector: Selected applications and lessons learned.
Triantafillou, P. (2015). The political implications of performance management and evidence-based policymaking. The American Review of Public Administration, 45(2), 167-181. DOI: 10.1177/0275074013483872
Valle-Cruz, D., Criado, J. I., Sandoval-Almazán, R., & Ruvalcaba-Gomez, E. A. (2020). Assessing the public policy-cycle framework in the age of artificial intelligence: From agenda-setting to policy evaluation. Government Information Quarterly, 37(4), 101509. DOI: 10.1016/j.giq.2020.101509
Van der Voort, H. G., Klievink, A. J., Arnaboldi, M., & Meijer, A. J. (2019). Rationality and politics of algorithms. Will the promise of big data survive the dynamics of public decision making?. Government Information Quarterly, 36(1), 27-38. DOI: 10.1016/j.giq.2018.10.011
Van Noordt, C., & Misuraca, G. (2022). Exploratory insights on artificial intelligence for government in Europe. Social Science Computer Review, 40(2), 426-444. DOI: 10.1177/0894439320980449
Zhang, D., Pee, L. G., Pan, S. L., & Cui, L. (2022). Big data analytics, resource orchestration, and digital sustainability: A case study of smart city development. Government information quarterly, 39(1), 101626. DOI: 10.1016/j.giq.2021.101626
Zhang, Y., Xu, Y., Shang, L., & Rao, K. (2007). An investigation into health informatics and related standards in China. International journal of medical informatics, 76(8), 614-620. DOI: 10.1016/j.ijmedinf.2006.05.003

  • Receive Date 16 October 2023
  • Revise Date 22 October 2023
  • Accept Date 28 February 2024