The Black Box Problem in Administering Justice: Risks of Opaque Algorithms in Legal Decision-Making
DOI:
https://doi.org/10.59022/ijlp.331Keywords:
Black Box Algorithms, Algorithmic Opacity, Legal Decision-Making, Judicial Transparency, AI in the Judiciary, Automated Decision Systems, Algorithmic Accountability, Bias in AIAbstract
As artificial intelligence (AI) technologies become integrated into judicial systems globally, the “black box problem” has emerged as a critical concern. This term refers to the opacity of machine learning algorithms, where the internal reasoning behind outputs is not transparent to users. This paper explores the implications of the black box problem in the administration of justice, focusing on how opaque algorithmic systems may undermine fairness, accountability, and trust in judicial processes. Drawing from global case studies, expert interviews, and legal theory, the study identifies key risks and proposes safeguards for transparent and ethical AI deployment in courts. The findings highlight the tension between technological efficiency and fundamental legal principles, suggesting that explainable AI must be prioritized to preserve judicial integrity and public confidence in legal institutions.
References
Alexander, M. (2012). The New Jim Crow: Mass incarceration in the age of colorblindness. The New Press.
Angwin, J., Larson, J., Mattu, S., & Kirchner, L. (2016, May 23). Machine bias. ProPublica.
Babuta, A., Oswald, M., & Rinik, C. (2020). Machine learning algorithms and police decision-making: Legal, ethical and regulatory challenges. Royal United Services Institute.
Barocas, S., & Selbst, A. D. (2016). Big data's disparate impact. California Law Review, 104, 671–732.
Binns, R. (2018). Fairness in machine learning: Lessons from political philosophy. In Proceedings of the 2018 Conference on Fairness, Accountability, and Transparency (FAT).
Bovens, M., & Zouridis, S. (2002). From street‐level to system‐level bureaucracies: How information and communication technology is transforming administrative discretion and constitutional control. Public Administration Review, 62(2), 174–184.
Brayne, S. (2017). Big data surveillance: The case of policing. American Sociological Review, 82(5), 977–1008.
Burrell, J. (2016). How the machine ‘thinks’: Understanding opacity in machine learning algorithms. Big Data & Society, 3(1), 1–12. https://doi.org/10.1177/2053951715622512
Carnegie Mellon University. (2018, October 11). Amazon scraps secret artificial intelligence recruiting engine that showed biases against women. Machine Learning Department, Carnegie Mellon University. [https://www.ml.cmu.edu/news/news-archive/2016-2020/2018/october/amazon-scraps-secret-artificial-intelligence-recruiting-engine-that-showed-biases-against-women.html](https://www.ml.cmu.edu/news/news-archive/2016-2020/2018/october/amazon-scraps-secret-artificial-intelligence-recruiting-engine-that-showed-biases-against-women.html)
Cerda, L. (2024, November 12). Prometea: Revolutionizing judicial efficiency in Argentina through artificial intelligence powered legal solutions. Inter-American Law Review. [https://inter-american-law-review.law.miami.edu/prometea-revolutionizing-judicial-efficiency-in-argentina-through-artificial-intelligence-powered-legal-solutions/](https://inter-american-law-review.law.miami.edu/prometea-revolutionizing-judicial-efficiency-in-argentina-through-artificial-intelligence-powered-legal-solutions/)
Citron, D., & Pasquale, F. (2014). The scored society: Due process for automated predictions. Washington Law Review, 89(1), 1–33.
Contini, F. (2020). Artificial intelligence and the transformation of humans, law and technology interactions in judicial proceedings. Law, Technology and Humans, 2(1), 4–18. [https://doi.org/10.5204/lthj.v2i1.1478](https://doi.org/10.5204/lthj.v2i1.1478)
Corvalán, J. (n.d.). Prometea: Artificial intelligence to transform justice and public organizations. [https://www.academia.edu/84557698/Prometea_Artificial_intelligence_to_transform_justice_and_public_organizations](https://www.academia.edu/84557698/Prometea_Artificial_intelligence_to_transform_justice_and_public_organizations)
Dias, S. A. de J., & Sátiro, R. M. (2024). Artificial intelligence in the judiciary: A critical view. Futures, 164, 103493. [https://doi.org/10.1016/j.futures.2024.103493](https://doi.org/10.1016/j.futures.2024.103493)
European Commission. (2021). Ethics guidelines for trustworthy AI. [https://digital-strategy.ec.europa.eu](https://digital-strategy.ec.europa.eu)
Gao, X. (2019). Chinese justice in the era of artificial intelligence. Journal of Zhejiang University (Humanities and Social Sciences), 49(4), 229–238. (https://doi.org/10.3785/j.issn.1008 942X.2019.04.02](https://doi.org/10.3785/j.issn.1008-942X.2019.04.02)
Garrett, B., & Stevenson, M. (2024). Algorithmic deference: An empirical analysis of judicial decision-making. Yale Law Journal, 133(4), 1021–1078.
Garrett, B. L., & Monahan, J. (2019). Judging risk. California Law Review, 108, 439–494.
Harcourt, B. E. (2007). Against prediction: Profiling, policing, and punishing in an actuarial age. University of Chicago Press.
Hannah-Moffat, K. (2019). Algorithmic risk governance: Big data analytics, race and information activism in criminal justice debates. Theoretical Criminology, 23(4), 453–470.
Kamarinou, D., Millard, C., & Singh, J. (2024). Algorithm-assisted decision-making in the public sector: A comparative study of three jurisdictions. Public Law, 2024(1), 94–117.
Kostka, G. (2019). China's social credit systems and public opinion: Explaining high levels of approval. New Media & Society, 21(7), 1565–1593.
Kosinski, M. (2025, April 17). What is black box AI and how does it work? IBM. (https://www.ibm.com/think/topics/black-box-ai)
Ma, L., Christensen, T., & Zheng, Y. (2023). Government technological capacity and public–private partnerships regarding digital service delivery: Evidence from Chinese cities. International Review of Administrative Sciences.
MacKenzie, D., & Wajcman, J. (Eds.). (2019). The social shaping of technology. Open University Press.
OECD. (2021). The impact of AI on the judicial system: Challenges and opportunities. OECD Publishing. https://www.oecd.org/gov/the-impact-of-ai-on-the-judicial-system.htm
Oswald, M., Grace, J., Urwin, S., & Barnes, G. C. (2018). Algorithmic risk assessment policing models: Lessons from the Durham HART model and “experimental” proportionality. Information & Communications Technology Law, 27(2), 223–250.
Papagianneas, S. (2022). Towards smarter and fairer justice? A review of the Chinese scholarship on building smart courts and automating justice. Global Perspectives, 3(1). [https://doi.org/10.1177/18681026211021412](https://doi.org/10.1177/18681026211021412)
Pasquale, F. (2015). The black box society: The secret algorithms that control money and information. Harvard University Press.
Peng, J., & Xiang, W. (2020). The rise of smart courts in China: Opportunities and challenges to the judiciary in a digital age. NAVEIÑ REET: Nordic Journal of Law and Social Research, 1(9), 345–372. [https://doi.org/10.7146/nnjlsr.v1i9.122167](https://doi.org/10.7146/nnjlsr.v1i9.122167)
Perry, W. L., McInnis, B., Price, C. C., Smith, S. C., & Hollywood, J. S. (2013). Predictive policing: The role of crime forecasting in law enforcement operations. RAND Corporation.
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016). SQuAD: 100,000+ questions for machine comprehension of text. arXiv. https://arxiv.org/abs/1606.05250
Ranchordás, S. (2021). Experimental legislation and the law. In Research Handbook on Experimental Governance (pp. 156–178). Edward Elgar Publishing.
Reiling, D. (2020). Courts and artificial intelligence. International Association for Court Administration.
Roberts, H., Cowls, J., Morley, J., Taddeo, M., Wang, V., & Floridi, L. (2021). The Chinese approach to artificial intelligence: An analysis of policy, ethics, and regulation. AI & Society, 36(1), 59–77.
Singh, J., Cobbe, J., & Lee, M. S. A. (2023). Recursive bias in justice: When algorithms learn from algorithmically influenced decisions. In Proceedings of the 2023 Conference on Fairness, Accountability, and Transparency (pp. 112–124).
Susskind, R. (2017). Tomorrow’s lawyers: An introduction to your future. Oxford University Press.
State v. Loomis, 881 N.W.2d 749 (Wis. 2016). https://law.justia.com/cases/wisconsin/supreme-court/2016/2015ap000157-cr.html
Vainsalu, H. (2021). AI in the public sector: Experiences from Estonia. e-Governance Academy Report.
Wang, N. (2020). “Black box justice”: Robot judges and AI-based judgment processes in China’s court system. In 2020 IEEE International Symposium on Technology and Society (ISTAS) (pp. 58–65). IEEE. [https://doi.org/10.1109/ISTAS50296.2020.9462216](https://doi.org/10.1109/ISTAS50296.2020.9462216)
World Justice Project. (2024). Global insights on legal technology: Annual report 2024. World Justice Project Publications.
Zeleznikow, J. (2017). Can artificial intelligence and online dispute resolution enhance efficiency and effectiveness in courts? International Journal for Court Administration, 8(2), 30–45.
Zuboff, S. (2019). The age of surveillance capitalism: The fight for a human future at the new frontiers of power. PublicAffairs.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Sojida Murodova
This work is licensed under a Creative Commons Attribution 4.0 International License.
