Risk Management in Agile Al/Ml Projects: Identifying and Mitigating Data and Model Risks
DOI:
https://doi.org/10.47941/jts.1824Keywords:
Machine Learning, Risk Perceptions, Management Tactics, Circumstances, Conference ProceedingsAbstract
Purpose: This study addresses the crucial challenge of managing risks associated with data and models in Agile Artificial Intelligence (AI) and Machine Learning (ML) projects. It aims to develop a systematic framework for effective risk control utilizing agile methodologies.
Methodology: The research is grounded in an interpretivist approach and utilizes a deductive method. It constructs a comprehensive framework for identifying and mitigating risks, integrating risk management seamlessly into Agile processes for AI and ML development.
Findings: The study introduces four technological themes critical for risk mitigation: dynamic distribution of resources, model robustness, risk integration, and quality assessment of information. These themes provide actionable strategies for reducing risks throughout the Agile AI/ML development lifecycle, ensuring that risk assessment and mitigation are integral to project planning and execution.
Unique contribution to theory, practice, and policy: The study contributes to both theory and practice by offering a detailed, actionable framework for risk management in Agile AI/ML projects. It advocates for the adoption of adaptive technologies and tools, continuous stakeholder engagement, and adherence to ethical standards. Recommendations include validation of the framework through empirical research and ongoing longitudinal evaluations to adapt and refine risk management practices. This approach aims to enhance the reliability and efficiency of project outputs in dynamic environments, providing a significant foundation for policy development in technology project management.
Downloads
References
Boukherouaa, E.B., Shabsigh, M.G., AlAjmi, K., Deodoro, J., Farias, A., Iskender, E.S., Mirestean, M.A.T. and Ravikumar, R., 2021. Powering the Digital Economy: Opportunities and Risks of Artificial Intelligence in Finance. International Monetary Fund.
AlAjmi, K., Deodoro, J., Farias, A., Iskender, E.S., Mirestean, A.T. and Ravikumar, R., Powering the Digital Economy.
Mattsson, U., 2023. Controlling Privacy and the Use of Data Assets-Volume 2: What is the New
World Currency–Data or Trust?. CRC Press.
Lennerz, J.K., Salgado, R., Kim, G.E., Sirintrapun, S.J., Thierauf, J.C., Singh, A., Indave, I.,
Bard, A., Weissinger, S.E., Heher, Y.K. and de Baca, M.E., 2023. Diagnostic quality model
(DQM): an integrated framework for the assessment of diagnostic quality when using AI/ML.
Clinical Chemistry and Laboratory Medicine (CCLM), 61(4), pp.544-557.
Verleyen, W. and McGinnis, W., 2022. Framework for disruptive AI/ML Innovation. arXiv
preprint arXiv:2204.12641.
Ashraf, I., Zikria, Y.B., Garg, S., Park, Y., Kaddoum, G. and Singh, S., 2022. Zero touch
networks to realize virtualization: Opportunities, challenges, and future prospects. IEEE Network,
(6), pp.251-259.
McGinnis, W., 2022. Framework for disruptive AI/ML Innovation. arXiv preprint
arXiv:2204.12641.
Novak, W. and CARNEGIE-MELLON UNIV PITTSBURGH PA, 2021. Artificial Intelligence
(AI) and Machine Learning (ML) Acquisition and Policy Implications.
Trujillo, S., Sabuj, S.R. and Bartels, R., 2023. University Medical Center Utrecht, Utrecht
University, Utrecht, Netherlands, 5 Department of Medical Microbiology, University Medical
Center Utrecht, Utrecht University, Utrecht, Netherlands. Surfacing best practices for AI software
development and integration in healthcare, p.14.
Elbegzaya, T., 2020. Application AI in Traditional Supply Chain Management DecisionMaking.
Millar, G., Kafchitsas, A., CLS, O.M., Kourtis, A., Xilouris, G., Christopoulou, M.,
Kolometsos, S., de Oca, E.M., MI, H.N.N., Pastor, A. and Fernandez, S., 2019. INtelligent security
and pervasive trust for 5G and beyond. Eur. Commission, Germany, Tech. Rep. H2020-EU, 2(1).
Krishankumar, R. and Ecer, F., 2023. Selection of IoT service provider for sustainable
transport using q-rung orthopair fuzzy CRADIS and unknown weights. Applied Soft Computing,
, p.109870.
Luo, M., 2022, December. Future Lies with Edge Computing. In International Conference on
Edge Computing (pp. 80-106). Cham: Springer Nature Switzerland.
Ashok, B. and Srivastava, S., 2021. LEADING THROUGH DISRUPTIONS A
DISTRIBUTED LEADERSHIP MODEL FOR BANKS IN INDIA. India Banking and Finance
Report 2021, p.3.
Carlsen, A., 2020. A self-assessment model for intelligent predictive maintenance analytics:
An update of the Analyses Module in the Maintenance Baseline Study (Master's thesis, University
of Stavanger, Norway).
Andresky, N. and Taliaferro, A., 2019. Operationalizing artificial intelligence for multidomain operations. Tech. Rep.
Huang, Y. and Chen, Y., 2020. Autonomous driving with deep learning: A survey of state-ofart technologies. arXiv preprint arXiv:2006.06091.
Jägare, U., 2019. Data science strategy for dummies. John Wiley & Sons.
Ahrens, J., Boehnlein, A., Carlson, R., Elliot, J., Fagnan, K., Ferrier, N., Foster, I., Gimpel,
L., Shalf, J. and Ratner, D., 2022. Envisioning Science in 2050. USDOE Office of Science
(SC)(United States).
Zaichkowsky, T.M., 2020. Systems architecture perspective on digital transformation for
financial services (Doctoral dissertation, Massachusetts Institute of Technology).
Hamidi, A., 2021. Machine Learning Engineering: An Exploratory Study of Challenges and
Practices (Doctoral dissertation, Ecole Polytechnique, Montreal (Canada)).
McGrath, M.J., Kennedy, J., Jensen, J., Crompton, S., Queralt, A., Dizdarevic, J., Stanovnik,
S., Černivec, A., Bizjak, M., Modic, J. and Font, R.S., D2. 3 Tracking Scientific, Technology and
Business Trends (Version 3).
Mazhar, N., Salleh, R., Hossain, M.A. and Zeeshan, M., 2020. SDN based intrusion detection
and prevention systems using manufacturer usage description: A survey. International Journal of
Advanced Computer Science and Applications, 11(12).
Da Silva, M., Flood, C.M. and Herder, M., 2022. Regulation of health-related artificial
intelligence in medical devices: the Canadian story. UBCL Rev., 55, p.635.
Hurtado Requena, S.J., 2023. Integration and analysis of biomedical data from multiple
clinical trials.
Stanton, B., Cirincione, G., Pham, T., Fischer, G., Ross, S., Whipps, G., Wiegmann, D. and
Bennett, K., 2020. US Army Artificial Intelligence Innovation Institute (A2I2) Aiding MultiDomain Operations (MDO). CCDC Army Research Laboratory Adelphi United States.
Kesharwani, S., 2019. Proclamation from Editor’s Inscription Counter. CYBERNOMICS, 1(7),
pp.1-4.
Rainie, L., Anderson, J. and Vogels, E.A., 2021. Experts doubt ethical AI design will be
broadly adopted as the norm within the next decade. Pew Research Center, 16.
Pierre, A., 2022. Leadership of Human-Machine Teams in Military Environments: An
Exploratory Framework (Doctoral dissertation, Regent University).
Cavalcante, I.M., Frazzon, E.M., Forcellini, F.A. and Ivanov, D., 2019. A supervised machine
learning approach to data-driven simulation of resilient supplier selection in digital manufacturing.
International Journal of Information Management, 49, pp.86-97
Sharma, K., Qian, F., Jiang, H., Ruchansky, N., Zhang, M. and Liu, Y., 2019. Combating fake
news: A survey on identification and mitigation techniques. ACM Transactions on Intelligent
Systems and Technology (TIST), 10(3), pp.1-42.
Ivanov, D., Dolgui, A., Das, A. and Sokolov, B., 2019. Digital supply chain twins: Managing
the ripple effect, resilience, and disruption risks by data-driven optimization, simulation, and
visibility. Handbook of ripple effects in the supply chain, pp.309-332.
Castro, D.C., Walker, I. and Glocker, B., 2020. Causality matters in medical imaging. Nature
Communications, 11(1), p.3673.
Obrenovic, B., Du, J., Godinic, D., Tsoy, D., Khan, M.A.S. and Jakhongirov, I., 2020.
Sustaining enterprise operations and productivity during the COVID-19 pandemic:“Enterprise
Effectiveness and Sustainability Model”. Sustainability, 12(15), p.5981.
Sharma, R., Shishodia, A., Kamble, S., Gunasekaran, A. and Belhadi, A., 2020. Agriculture
supply chain risks and COVID-19: mitigation strategies and implications for the practitioners.
International Journal of Logistics Research and Applications, pp.1-27.
Hotle, S., Murray-Tuite, P. and Singh, K., 2020. Influenza risk perception and travel-related
health protection behavior in the US: Insights for the aftermath of the COVID-19 outbreak.
Transportation Research Interdisciplinary Perspectives, 5, p.100127.
Thiebes, S., Lins, S. and Sunyaev, A., 2021. Trustworthy artificial intelligence. Electronic
Markets, 31, pp.447-464.
Realmonte, G., Drouet, L., Gambhir, A., Glynn, J., Hawkes, A., Köberle, A.C. and Tavoni,
M., 2019. An inter-model assessment of the role of direct air capture in deep mitigation pathways.
Nature communications, 10(1), p.3277.
Bommasani, R., Hudson, D.A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M.S.,
Bohg, J., Bosselut, A., Brunskill, E. and Brynjolfsson, E., 2021. On the opportunities and risks of
foundation models. arXiv preprint arXiv:2108.07258.
Amanullah, M.A., Habeeb, R.A.A., Nasaruddin, F.H., Gani, A., Ahmed, E., Nainar, A.S.M.,
Akim, N.M. and Imran, M., 2020. Deep learning and big data technologies for IoT security.
Computer Communications, 151, pp.495-517.
Tang, B., Wang, X., Li, Q., Bragazzi, N.L., Tang, S., Xiao, Y. and Wu, J., 2020. Estimation
of the transmission risk of the 2019-nCoV and its implication for public health interventions.
Journal of clinical medicine, 9(2), p.462.
World Health Organization, 2021. Strategic toolkit for assessing risks: a comprehensive toolkit
for all-hazards health emergency risk assessment.
Gerry, S., Bonnici, T., Birks, J., Kirtley, S., Virdee, P.S., Watkinson, P.J. and Collins, G.S.,
Early warning scores for detecting deterioration in adult hospital patients: systematic review
and critical appraisal of methodology. bmj, 369.
Sarker, I.H., Kayes, A.S.M., Badsha, S., Alqahtani, H., Watters, P. and Ng, A., 2020.
Cybersecurity data science: an overview from machine learning perspective. Journal of Big data,
, pp.1-29.
Ngo, P.T.T., Panahi, M., Khosravi, K., Ghorbanzadeh, O., Kariminejad, N., Cerda, A. and
Lee, S., 2021. Evaluation of deep learning algorithms for national scale landslide susceptibility
mapping of Iran. Geoscience Frontiers, 12(2), pp.505-519.
Burgos, D. and Ivanov, D., 2021. Food retail supply chain resilience and the COVID-19
pandemic: A digital twin-based impact analysis and improvement directions. Transportation
Research Part E: Logistics and Transportation Review, 152, p.102412.
Anderegg, W.R., Trugman, A.T., Badgley, G., Anderson, C.M., Bartuska, A., Ciais, P.,
Cullenward, D., Field, C.B., Freeman, J., Goetz, S.J. and Hicke, J.A., 2020. Climate-driven risks
to the climate mitigation potential of forests. Science, 368(6497), p.eaaz7005.
Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A.,
Brown, T., Song, D., Erlingsson, U. and Oprea, A., 2021. Extracting training data from large
language models. In 30th USENIX Security Symposium (USENIX Security 21) (pp. 2633-2650).
Rathore, S., Kwon, B.W. and Park, J.H., 2019. BlockSecIoTNet: Blockchain-based
decentralized security architecture for IoT network. Journal of Network and Computer
Applications, 143, pp.167-177.
Alegbeleye, O.O. and Sant’Ana, A.S., 2020. Manure-borne pathogens as an important source
of water contamination: An update on the dynamics of pathogen survival/transport as well as
practical risk mitigation strategies. International journal of hygiene and environmental health, 227,
p.113524.
Sellgren, C.M., Gracias, J., Watmuff, B., Biag, J.D., Thanos, J.M., Whittredge, P.B., Fu, T.,
Worringer, K., Brown, H.E., Wang, J. and Kaykas, A., 2019. Increased synapse elimination by
microglia in schizophrenia patient-derived models of synaptic pruning. Nature neuroscience,
(3), pp.374-385.
Scher, J.U., Ogdie, A., Merola, J.F. and Ritchlin, C., 2019. Preventing psoriatic arthritis: focusing on patients with psoriasis at increased risk of transition. Nature Reviews Rheumatology, 15(3), pp.153-166.
HaddadPajouh, H., Dehghantanha, A., Parizi, R.M., Aledhari, M. and Karimipour, H., 2021. A survey on internet of things security: Requirements, challenges, and solutions. Internet of Things, 14, p.100129.
Hosseini, S., Morshedlou, N., Ivanov, D., Sarder, M.D., Barker, K. and Al Khaled, A., 2019. Resilient supplier selection and optimal order allocation under disruption risks. International Journal of Production Economics, 213, pp.124-137.
Shuja, J., Alanazi, E., Alasmary, W. and Alashaikh, A., 2021. COVID-19 open source data sets: a comprehensive survey. Applied Intelligence, 51, pp.1296-1325.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Ankur Tak, Sunil Chahal
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution (CC-BY) 4.0 License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
