A systematic literature review on transparency and interpretability of AI models in healthcare: taxonomies, tools, techniques, datasets, open research challenges, and future trends

Mridha K, Uddin MM, Shin J, Khadka S, Mridha MF. An interpretable skin cancer classification using optimized convolutional neural network for a smart healthcare system. IEEE Access, 2023.

Shaban-Nejad A, Michalowski M, Brownstein JS, Buck- eridge DL. Guest editorial explainable ai: towards fairness, accountability, transparency and trust in healthcare. IEEE J Biomed Health Inf. 2021;25(7):2374–5.

Article  Google Scholar 

Cutillo CM, Sharma KR, Foschini L, Kundu S, Mackintosh M, Mandl KD. Machine intelligence in healthcare—perspectives on trustworthiness, explainability, usability, and transparency. NPJ Digit Med. 2020;3(1):47. and M. in Healthcare Workshop Working Group Beck Tyler 1 Collier Elaine 1 Colvis Christine 1 Gersing Kenneth 1 Gordon Valery 1 Jensen Roxanne 8 Shabestari Behrouz 9 Southall Noel 1.

Jiang F, Jiang Y, Zhi H, Dong Y, Li H, Ma S, Wang Y, Dong Q, Shen H, Wang Y. Artificial intelligence in healthcare: past, present and future. Stroke Vascular Neurol. 2017;2(4).

El Naqa I, Karolak A, Luo Y, Folio L, Tarhini AA, Rollison D, Parodi K. Translation of AI into oncology clinical practice. Oncogene. 2023;42(42):3089–97.

Article  Google Scholar 

Reyes M, Meier R, Pereira S, Silva CA, Dahlweid F-M, v. Tengg-Kobligk H, et al. On the interpretability of artificial intelligence in radiology: challenges and opportunities. Radiol Artif Intell. 2020;2(3):e190043.

Article  Google Scholar 

Hanif AM, Beqiri S, Keane PA, Campbell JP. Applications of interpretability in deep learning models for ophthalmology. Curr Opin Ophthalmol. 2021;32(5):452.

Article  Google Scholar 

Saraswat D, Bhattacharya P, Verma A, Prasad VK, Tanwar S, Sharma G, Bokoro PN, Sharma R. Explainable Ai for healthcare 5.0: opportunities and challenges. IEEE Access. 2022;10:84486–517.

Article  Google Scholar 

Bhagya J, Thomas J, Raj ED. Exploring explainability and transparency in deep neural networks: A comparative approach, in 2023 7th International Conference on Intelligent Computing and Con- trol Systems (ICICCS), IEEE; 2023. pp. 664–669.

Mishra P. Model explainability and interpretability. In: Practical Ex- plainable AI Using Python: Artificial Intelligence Model Explanations Using Python-based Libraries, Extensions, and Frameworks. Springer; 2021. p. 1–22.

Google Scholar 

Wang B, Jing J, Huang X, Hua C, Qin Q, Jia Y, et al. Establishment of a knowledge-and-data-driven artificial intelligence system with robustness and interpretability in laboratory medicine. Adv Intell Syst. 2022;4(5):2100204.

Article  Google Scholar 

Joyce DW, Kormilitzin A, Smith KA, Cipriani A. Explainable artificial intelligence for mental health through transparency and interpretability for understandability. NPJ Digit Med. 2023;6(1):6.

Article  Google Scholar 

Salahuddin Z, Woodruff HC, Chatterjee A, Lambin P. Trans- parency of deep neural networks for medical image analysis: a review of interpretability methods. Comput Biol Med. 2022;140:105111.

Article  Google Scholar 

Fernandez-Quilez A. Deep learning in radiology: ethics of data and on the value of algorithm transparency, interpretability and explainability. AI Ethics. 2023;3(1):257–65.

Article  Google Scholar 

MacDonald S, Steven K, Trzaskowski M. Interpretable ai in healthcare: Enhancing fairness, safety, and trust. In: Artificial Intelli- gence in Medicine: Applications, Limitations and Future Directions. Springer; 2022. p. 241–58.

Chapter  Google Scholar 

Ennab M, Mcheick H. Designing an interpretability-based model to explain the artificial intelligence algorithms in healthcare. Diagnostics. 2022;12(7):1557.

Article  Google Scholar 

Winter PD, Carusi A. (de) troubling transparency: artificial intelligence (ai) for clinical applications. Med Humanit. 2023;49(1):17–26.

Article  Google Scholar 

Va¨a¨na¨nen A, Haataja K, Vehvila¨inen-Julkunen K, Toivanen P. Ai in healthcare: a narrative review. F1000Res. 2021;10:6.

Article  Google Scholar 

Amann J, Blasimme A, Vayena E, Frey D, Madai VI, Consortium P. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC Med Inf Decis Mak. 2020;20:1–9.

Google Scholar 

Saraswat D, Bhattacharya P, Verma A, Prasad VK, Tanwar S, Sharma G, Bokoro PN, Sharma R. Explainable Ai for healthcare 5.0: opportunities and challenges. IEEE Access; 2022.

Jungmann F, Ziegelmayer S, Lohoefer FK, Metz S, M¨uller- Leisse C, Englmaier M, Makowski MR, Kaissis GA, Braren RF. Algorithmic transparency and interpretability measures im- prove radiologists’ performance in bi-rads 4 classification. Eur Radiol. 2023;33(3):1844–51.

Article  Google Scholar 

Smith KA, Cipriani A. Explainable artificial intelligence for mental health through transparency and interpretability for understand- ability, 2023.

Xu Q, Xie W, Liao B, Hu C, Qin L, Yang Z, Xiong H, Lyu Y, Zhou Y, Luo A et al. Interpretability of clinical decision support systems based on artificial intelligence from technological and medical perspective: A systematic review. J Healthc Eng. 2023;2023.

Neri E, Aghakhanyan G, Zerunian M, Gandolfo N, Grassi R, Miele V, Giovagnoni A, Laghi A. and S. Expert group on Artifi- Cial Intelligence, explainable Ai in radiology: a white paper of the Italian society of medical and interventional radiology. Radiol Med. 2023:1–10.

Swathi Y, Challa M. A comparative analysis of explainable ai techniques for enhanced model interpretability, in 2023 3rd Inter- national Conference on Pervasive Computing and Social Networking (ICPCSN), IEEE; 223. pp. 229–234.

Band SS, Yarahmadi A, Hsu C-C, Biyari M, Sookhak M, Ameri R, et al. Applica- tion of explainable artificial intelligence in medical health: a systematic review of interpretability methods. Inf Med Unlocked. 2023. https://doi.org/10.1016/j.imu.2023.101286.

Article  Google Scholar 

Sinha A, Garcia DW, Kumar B, Banerjee P. Application of big data analytics and internet of medical things (iomt) in healthcare with view of explainable artificial intelligence: A survey. In: Interpretable Cognitive Internet of Things for Healthcare. Springer; 2023. p. 129–63.

Chapter  Google Scholar 

Verma M. Artificial intelligence role in modern science: Aims, merits, risks and its applications. Artif Intell. 2023;7(5).

Curia F. Explainable and transparency machine learning approach to predict diabetes develop. Health Technol. 2023;1–12.

Venkatesh S, Narasimhan K, Adalarasu K et al. An overview of interpretability techniques for explainable artificial intelligence (xai) in deep learning-based medical image analysis, in 2023 9th International Conference on Advanced Computing and Communication Systems (ICACCS), IEEE; 2023. vol. 1, pp. 175–182.

Bharati S, Mondal MRH, Podder P. A review on explainable artificial intelligence for healthcare: Why, how, and when? IEEE Trans Artif Intell, 2023.

Kurek W, Pawlicki M, Pawlicka A, Kozik R, Choras´ M. Explainable artificial intelligence 101: Techniques, applications and challenges. In: International Conference on Intelligent Computing, Springer, 2023. pp. 310–318.

Hassija V, Chamola V, Mahapatra A, Singal A, Goel D, Huang K, et al. Interpreting black-box models: a review on explainable artificial intelligence. Cogn Comput. 2023;(1):30. https://doi.org/10.1007/s12559-023-10179-8.

Pal M, Mistry S, De D. Interpretability approaches of explain- able ai in analyzing features for lung cancer detection. In: Frontiers of ICT in Healthcare: Proceedings of EAIT 2022. Springer; 2023. p. 277–87.

Chapter  Google Scholar 

Gonza´lez-Alday R, Garc´ıa-Cuesta E, Kulikowski CA, Maojo V. A scoping review on the progress, applicability, and future of ex- plainable artificial intelligence in medicine. Appl Sci. 2023;13(19):10778.

Article  Google Scholar 

Embarak O. Decoding the black box: A comprehensive review of ex- plainable artificial intelligence, in 2023 9th International Conference on Information Technology Trends (ITT), IEEE; 2023. pp. 108–113.

Husnain A, Rasool S, Saeed A, Hussain HK. Revolutionizing pharmaceutical research: harnessing machine learning for a paradigm shift in drug discovery. Int J Multidisciplinary Sci Arts. 2023;2(2):149–57.

Article  Google Scholar 

de Souza Jr LA, Mendel R, Strasser S, Ebigbo A, Probst A, Messmann H, Papa JP, Palm C. Convolutional Neural Networks for the evaluation of cancer in Barrett’s esophagus: Explainable AI to lighten up the black-box. PhD thesis, 2021.

Bernal J, Mazo C. Transparency of artificial intelligence in healthcare: insights from professionals in computing and healthcare worldwide. Appl Sci. 2022;12(20):10228.

Article  Google Scholar 

Zhang Z, Genc Y, Wang D, Ahsen ME, Fan X. Effect of ai explanations on human perceptions of patient-facing ai-powered healthcare systems. J Med Syst. 2021;45(6):64.

Article  Google Scholar 

Kolyshkina I, Simoff S. Interpretability of machine learning solutions in public healthcare: the crisp-ml approach. Front Big Data. 2021;4:660206.

Article  Google Scholar 

Stenwig E, Salvi G, Rossi PS, Skjærvold NK. Comparative analysis of explainable machine learning prediction models for hospital mortality. BMC Med Res Methodol. 2022;22(1):53.

Article  Google Scholar 

Jin D, Sergeeva E, Weng W-H, Chauhan G, Szolovits P. Explainable deep learning in healthcare: a methodological survey from an attribution view. WIREs Mech Dis. 2022;14(3):e1548.

Article  Google Scholar 

Al Kuwaiti A, Nazer K, Al-Reedy A, Al-Shehri S, Al-Muhanna A, Subbarayalu AV, Al Muhanna D, Al-Muhanna FA. A review of the role of artificial intelligence in healthcare. J Personalized Med. 2023;13(6): 951.

Article  Google Scholar 

Das S, Agarwal N, Venugopal D, Sheldon FT, Shiva S. Taxonomy and survey of interpretable machine learning method, in 2020 IEEE Symposium Series on Computational Intelligence (SSCI), IEEE; 2020. pp. 670–677.

Payrovnaziri SN, Chen Z, Rengifo-Moreno P, Miller T, Bian J, Chen JH, Liu X, He Z. Explainable artificial intelligence models using real-world electronic health record data: a systematic scoping review. J Am Med Inform Assoc. 2020;27(7):1173–85.

Article  Google Scholar 

Abdullah TA, Zahid MSM, Ali W. A review of interpretable Ml in healthcare: taxonomy, applications, challenges, and future direc- tions. Symmetry. 2021;13(12):2439.

Article  Google Scholar 

Linardatos P, Papastefanopoulos V, Kotsiantis S. Explainable ai: a review of machine learning interpretability methods. Entropy. 2020;23(1):18.

Article  Google Scholar 

Rong G, Mendez A, Assi EB, Zhao B, Sawan M. Artificial intelligence in healthcare: review and prediction case studies. Engineering. 2020;6(3):291–301.

Article  Google Scholar 

Stiglic G, Kocbek P, Fijacko N, Zitnik M, Verbert K, Cilar L. Interpretability of machine learning-based prediction models in health- care. WIREs Data Min Knowl Discov. 2020;10(5):e1379.

Article  Google Scholar 

Markus AF, Kors JA, Rijnbeek PR. The role of explainability in creating trustworthy artificial intelligence for health care: a com- prehensive survey of the terminology, design choices, and evaluation strategies. J Biomed Inform. 2021;113:103655.

Article  Google Scholar 

Arrieta AB, D´ıaz-Rodr´ıguez N, Del Ser J, Bennetot A, Tabik S, Barbado A, Garc´ıa S, Gil-Lo´pez S, Molina D, Benjamins R, et al. Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible Ai. Inform Fusion. 2020;58:82–115.

Article  Google Scholar 

Reader TW, Gillespie A, Roberts J. Patient complaints in healthcare systems: a systematic review and coding taxonomy. BMJ Qual Saf. 2014;23(8):678–89.

Article  Google Scholar 

Karmani P, Chandio AA, Karmani V, Korejo IA, Chandio MS. Taxonomy on healthcare system based on machine learning. Int J Comput Digit Syst. 2020;9(6):1199–212.

Article  Google Scholar 

Speith T. A review of taxonomies of explainable artificial intelligence (xai) methods, in Proceedings of the 2022 ACM conference on fairness, accountability, and transparency, 2022. pp. 2239–2250.

Rakovska MA, Stratieva SV. A taxonomy of healthcare supply chain management practices. In: Supply Chain Forum: An International Journal. Taylor & Francis; 2018. vol. 19, pp. 4–24.

Aceto G, Persico V, Pescape´ A. The role of information and communication technologies in healthcare: taxonomies, perspectives, and challenges. J Netw Comput Appl. 2018;107:125–54.

Article  Google Scholar 

Hammoda B, Durst S. A taxonomy of knowledge risks for healthcare organizations. VINE J Inf Knowl Manag Syst. 2022;52(3):354–72.

Google Scholar 

Shamshirband S, Fathi M, Dehzangi A, Chronopoulos AT, Alinejad-Rokny H. A review on deep learning approaches in health- care systems: taxonomies, challenges, and open issues. J Biomed Inform. 2021;113:103627.

Article  Google Scholar 

Comments (0)

No login
gif