Here is an interesting topic on AI – the Risk and Safety Issues associated with Machine Learning (ML). AI and AIPPS issues continue to draw public and professional attentions – because of their high impacts – proliferating in both good and bad ways. They are already occurring and more are on the horizon.
Needless to say, among others, the profession of Engineering is part of it.
Some are highlighted in Artificial Intelligence and Uncertainty and Risk, and in different ASCE Collaborate Threads.
I am prompted to float this topic while reading a NAP Task Committee Pre-Publication Copy: 2025 NAP #27970 – Machine Learning for Safety-Critical Applications . . . Have a look.
As with other NAP publications I have read and quoted – this one also doesn't disappoint. It basically describes the state-of-art of AI systems and related issues to help the committee generate tasks (see Appendix A – Statement of Task).
By using the term Safety-Critical – the authors rightly set the direction to focusing on those AIPPS – that are significantly vulnerable than others in risk factors and associated safety issues.
As we have discussed before: AI, an Informatics System of Adaptive Algorithms & Routines, is data and model driven – and how they are used to train the machines to learn – to let them generate intelligent answers to human queries and needs. This implies that – the quality, quantity, neutrality of data and model used to train the machine – and of generating answers . . . have a significant impact on the performance of AIPPS – the soundness, weakness or prejudices of them. Also important are the competence, integrity and traits of personnel – tasked to develop and market the system . . .
The authors used one word trustworthiness again and again. This emphasis tells how the committee intends to accomplish their tasks. Hope, trustworthiness of data and models used to train the AI machine – would come into the purview of critical screening – as they are the sources of most safety issues.
--------------------
Dr. Dilip K Barua, PhD
Website Links and Profile
-------------------------------------------