"Explainable AI" represents a growing area of interest in the field of artificial intelligence and machine learning. While AI technologies can automatically make decisions based on data, "Explainable AI" also provides individual reasons for these decisions, helping to avoid the so-called "black box" phenomenon, in which AI arrives at conclusions through unclear and potentially problematic means. Although certain techniques can also provide hypothetical improvements that could be achieved when an undesirable outcome occurs, they do not offer any concrete steps for improvement. For example, if an AI makes judgments about a person's health status and determines that they are unhealthy, the new technology can be applied to first explain the reason for the result based on medical data such as height, weight, and blood pressure. It can then offer the user specific suggestions on how best to regain health, identifying the interaction between a large number of complex elements from the medical tests and previous data, showing specific steps for improvement, taking into account the feasibility and difficulty of implementation. This new technology offers the possibility of improving the transparency and reliability of decisions made by Artificial Intelligence, enabling more people to interact with these technologies in the future, providing a sense of trust and peace of mind. Background to the Development:
Currently, deep learning technologies widely used in AI systems requiring advanced tasks, such as facial recognition or autonomous driving, automatically make various decisions based on large amounts of data using a kind of “black box” predictive model. However, in the future, ensuring the transparency and reliability of Artificial Intelligence systems will become a crucial issue for AI to make fundamental contributions to society. This need has led to increased interest and research in “Explainable AI” technologies. For example, in medical assessments, AI can successfully determine the risk level of a disease based on data such as weight and muscle mass (Figure 1(A)). In addition to the risk assessment results, increasing attention has focused on the "explainable AI" that presents the attributes (Figure 1(B)) that formed the basis for the assessment. Since the AI determines that health risks are high based on the attributes of the input data, it is possible to change the values of these attributes to obtain the desired results with low health risks.
To achieve the desired results in AI-automated decisions, it is necessary not only to present the attributes that need to be changed, but also those that can be changed with the least possible effort.
In the case of medical checkups, if the goal is to change the AI decision outcome from a high-risk state to a low-risk one, achieving this with less effort might seem to be increasing muscle mass (Figure 2 Change 1). However, increasing only muscle mass without changing weight is unrealistic, so increasing both weight and muscle mass simultaneously is a more realistic solution (Figure 2 Change 2). Furthermore, there are many interactions between attributes like weight and muscle mass, such as causal relationships where weight increases with muscle growth, and the total effort required to make the changes depends on the order in which the attributes are changed. Therefore, it is necessary to present the appropriate order in which the attributes are changed. In Figure 2, it is not obvious whether weight or muscle mass should be modified first to reach Change 2 from the current state, so finding an appropriate method remains a challenge, considering the possibility and order of changes among a large number of potential candidates.
Fig. 2 Changes in attributes
About the new technology developed: Through joint research on machine learning and data mining, Fujitsu Laboratories and the Arimura Laboratory of the Graduate School of Information Science and Technology at Hokkaido University have developed new AI technologies that can explain to users the reasons behind AI decisions, enabling the discovery of useful and actionable insights.
AI technologies such as LIME (1) and SHAP (2), developed to support user decision-making, make decisions convincing by explaining the AI's reasoning behind them. The newly developed technology is based on the concept of counterfactual explanation (3) and presents attribute changes and their execution order as a procedure. While preventing unrealistic changes by analyzing past cases, the AI estimates the effects of attribute value changes on other values, such as causality, and calculates the actual amount the user needs to change. This allows the AI to present actions that will achieve optimal results in the correct order and with the least effort.
For example, if one needs to add 1 kg of muscle mass and 7 kg to their body weight to reduce the risk associated with the input attribute and its order (Figure 1(C)) that changes to achieve the desired result in a medical check-up, it is possible to estimate the relationship by analyzing the interaction between muscle mass and body weight beforehand. This means that if 1 kg of muscle mass is added, body weight will increase by 6 kg. In this case, of the additional 7 kg needed for the weight gain, the required amount of muscle mass is only 1 kg. In other words, the amount of change one actually needs to make is adding 1 kg of muscle mass and 1 kg of weight, so one can achieve the desired result with less effort compared to changing their weight first.
Fig.3 Interactions and changes between attributes
Grades
[1] LIME. An explainable AI technology. Explains in a simple and interpretable model.
[2] SHAP. An explainable AI technology. Showing the contribution of the explanatory variable in the model.
[3] Counterfactual explanation. A method for indicating and explaining a state different from the truth, such as "If I had done this, the result would have been."
