A comparison of causal discovery and explainable AI (XAI) for image datasets

The recent push for fair, trustworthy, and responsible Artificial Intelligence (AI) and Machine Learning (ML) systems have pushed for more explainable systems that are capable of explaining their predictions/decisions and inner workings. This led to the field of Explainable AI (XAI) going through an exponential growth in the past few years. XAI has been crucial in making AI/ML systems more comprehensible. However, XAI is limited to the model that it is being applied to, for both post-hoc or transparent models. Even though XAI can explain the decisions being made by the ML systems, these decisions are based on correlation and not causation. For applications such as tumor classification in the medical field, this can have serious consequences as people’s lives are affected. A potential solution for this challenge is the application of causal learning, which goes beyond the limitations of correlation for ML systems. Causal learning can generate analysis based on cause-and-effect relations within the data. This study compares the results of explanations given by post-hoc XAI systems to the causal features derived from causal graphs via causal discover for image datasets. We investigate how well XAI explanations/interpretations are able to identify the pertinent features within images. Causal graphs are generated for image datasets to extract the causal features that have a direct cause- and-effect relation with the label. These features are then compared to the features highlighted by XAI via feature relevance. The addition of causal learning for image datasets can aide in achieving fairness, bias detection, and mitigation to provide a robust and trustworthy system. We highlight the limitation of XAI tools such as LIME to make predictions based on physical features from images, whereas causal discovery can go beyond the simple pixel based perturbations to identify causal relations from image attributes.