Making decisions based on predictions generated from machine learning models requires users to have a clear understanding of the mechanisms and logic behind every prediction. From one side, business users must be convinced in the ability of the models to generate correct predictions. Predictive power, expressed by the different performance measures, is not sufficient for building trust and acceptance of machine learning models. Business users need additional techniques and tools for model interpretation and evaluation of the effects from decisions based on machine learning predictions. In this research paper we propose an approach for analyzing the impact of different thresholds for converting probabilities into predictions for binomial classification machine learning models applicable for credit risk prediction in Peer-to-Peer Lending platforms. We define a set of measures to explore global and local impact on decision-making process and present different scenarios in a Business Intelligence application built in an analytical and business intelligent platform. Based on the presented results we can draw conclusions that when choosing the best model and threshold users should consider a broad set of measures not only for model accuracy, but also should consider misclassification costs, financial results, asset portfolio structure, etc.
Aleksandrova, Y. (2021) Comparing Performance of Machine Learning Algorithms for Default Risk Prediction in Peer to Peer Lending, TEM Journal, 10(1), pp. 133–143. Available at: https://doi.org/10.18421/TEM101-16 [Accessed 01/05/2024].
Ariza-Garzon, M., Arroyo, J., Caparrini, A., & Segovia-Vargas, M.-J. (2020). Explainability of a Machine Learning Granting Scoring Model in Peer-to-Peer Lending. IEEE Access, 8, pp. 64873- 64890. Available at: doi:10.1109/ACCESS.2020.2984412 [Accessed 01/05/2024]
Been, K., Khanna, R., & Koyejo, O. (2016). Examples are not enough, learn to criticize! Criticism for interpretability. 29th Conference on Neural Information Processing Systems (NIPS 2016), 29, pp. 2280-2288. Barcelona, Spain.
Cambridge Center of Alternative Finance. (2023). The Global COVID-19 Fintech Market Impact and Industry Resilience Study. [Online] Available from: https://www.jbs.cam.ac.uk/wp- content/uploads/2022/06/2022-global-covid19-fintech-market-study.pdf [Accessed 01/05/2024]
Cetin, A.I. and Ahmed, S.E. (2023) Determinants of Credit Ratings and Comparison of the Rating Prediction Performances of Machine Learning Algorithms, E3S Web of Conferences, 409. Available at: https://doi.org/10.1051/e3sconf/202340905013 [Accessed 01/05/2024].
Frazier, P.I. (2018) A Tutorial on Bayesian Optimization. arXiv:1807.02811 [stat.ML]. Available at: https://doi.org/10.48550/arXiv.1807.02811 [Accessed 01/05/2024].
Gall, R. (2018). Machine Learning Explainability vs Interpretability: Two concepts that could help restore trust in AI. [Online] Available from: https://www.kdnuggets.com/2018/12/machine- learning-explainability-interpretability-ai.html [Accessed 01/05/2024].
Gutzwiller, K.J. and Chaudhary, A. (2020) Machine-learning models, cost matrices, and conservation-based reduction of selected landscape classification errors, Landscape Ecology, 35(2), pp. 249–255. Available at: https://doi.org/10.1007/s10980-020-00969-y [Accessed 01/05/2024].
Kehayova-Stoycheva, M. and Vasilev, J. (2023) Developing a Household Sustainable Consumption Index Calculator – Marketing and IT Dimensions, TEM Journal, 12(4), pp. 2101– 2111. Available at: https://doi.org/10.18421/TEM124-21 [Accessed 01/05/2024].
Lohani, B.P. et al. (2022) Machine Learning Based Model for Prediction of Loan Approval, Proceedings of 3rd International Conference on Intelligent Engineering and Management, ICIEM 2022, pp. 465–470. Available at: https://doi.org/10.1109/ICIEM54221.2022.9853160 [Accessed 01/05/2024]..
Lundberg, S., & Lee, S.-I. (2017). A Unified Approach to Interpreting Model Predictions. Advances in neural information processing systems, pp. 4765-4774 Available from: https://proceedings.neurips.cc/paper_files/paper/2017/file/8a20a8621978632d76c43dfd28b6776 7-Paper.pdf [Accessed 01/05/2024].
Meshref, H. (2020) Predicting loan approval of bank direct marketing data using ensemble machine learning algorithms, International Journal of Circuits, Systems and Signal Processing, 14, pp. 914–922. Available at: https://doi.org/10.46300/9106.2020.14.117 [Accessed 01/05/2024].
Miller, T. (2019). Explanation in Artificial Intelligence: Insights from the Social Sciences. Artificial Intelligence, 267, 1-38. Available from: doi:https://doi.org/10.1016/j.artint.2018.07.007 [Accessed 01/05/2024]
Molnar, C. (2020). Interpretable Machine Learning. A Guide for Making Black Box Models Explainable. [Online] Available from: https://christophm.github.io/interpretable-ml- book/index.html [Accessed 01/05/2024]
Petrov, P., Stoev, S., Radev, M., & Sergeev, A. (2021). RISK MANAGEMENT PROCESSES IN INFORMATION SYSTEMS DEVELOPMENT. International Multidisciplinary Scientific GeoConference Surveying Geology and Mining Ecology Management, SGEM, 21 (7), pp. 467- 473. Available from: doi:10.5593/sgem2021/2.1/s07.18 [Accessed 01/05/2024]
Schlegel, K., Sun, J., Pidsley, D., Ganeshan, A., Fei, F., & Popa, A. (2023). Magic Quadrant for Analytics and Business Intelligence Platforms. [Online] Available from https://www.gartner.com/doc/reprints?id=1-2955ETOT&ct=220215&st=
sb?ocid=lp_pg398450-_gdc_comm_az [Accessed 01/05/2024]
Setiawan, N., Suharjito and Diana (2019) A comparison of prediction methods for credit default on peer to peer lending using machine learning, Procedia Computer Science, 157, pp. 38–45. Available at: https://doi.org/10.1016/j.procs.2019.08.139 [Accessed 01/05/2024].
Zhang, C. and Zhou, X. (2023) Forecasting Credit Risk of SMEs in Supply Chain Finance Using Bayesian Optimization and XGBoost, Mathematical Problems in Engineering, 2023. Available at: https://doi.org/10.1155/2023/5609996 [Accessed 01/05/2024].
