Threat Modelling for ML/AI systems

Machine Learning and Artificial Intelligence have been attracting a lot of attention in the past few years, and since the release of ChatGPT in 2023, the interest in these topics has exploded. We are assisting to a gold rush towards these technologies and, amidst the rush, it is too easy to forget about security. One critical challenge in the realm of ML/AI is that models are often black-box. Understanding how these systems arrive at conclusions, evaluating their behaviour, and ensuring consistency across a large input space can be elusive. At Equinor, we have started adopting Threat Modelling practices to identify potential risks in the use of ML/AI systems. Threat Modelling is recognised as one of the most effective practices in improving security in software solutions.