Classification of Post-Hoc Explainability Techniques
Post-hoc explanation strategies in Explainable Artificial Intelligence (XAI) can be roughly divided into few categories they are as follows:
- Model-independent techniques
- Model-specific techniques
- Deep learning models
- Model Dependent Techniques
Model-Independent Techniques
1. Importance of Features: This strategy entails determining which features or input variables had the greatest influence on the model's conclusion. A number of techniques, including partial dependence graphs, SHAP values, and permutation importance, can be used to determine a feature's importance. This method can help determine which aspects are most important when making the choice and also shed light on how the model functions.
2. Local Explanations: Local explanations concentrate on describing the choices a model made in relation to a particular instance or input. To shed light on why the model chose a specific action for a particular input, techniques like LIME (Local Interpretable Model-Agnostic Explanations) and Anchors can be applied. This method can be very helpful for debugging and error analysis, and it can assist pinpoint areas that need improvement.
3. Global Explanations: Global explanations give a broad knowledge of the model's decision-making processes in relation to all inputs. To examine how each characteristic influences a model's choice, methods like SHAP (Shapley Additive Explanations) or ICE (Individual Conditional Expectation) plots can be employed. This method can be used to find patterns in the data and reveal how the model interprets information.
4. Counterfactual Explanations: These explanations show how altering the input attributes can affect the model's judgment. This method can assist in determining which aspects are most crucial for the choice and offer suggestions on how to enhance the model's functionality. To produce counterfactual examples and explain how the model would behave if the input features were different, methods like CEM (Counterfactual Explanations via Model Extraction) or CFM (Counterfactual Fairness Machine) might be employed.
5. Visualization: Using visualization tools can assist make it easier to comprehend how the model decides to act. The contribution of each feature to the model's decision-making process can be seen using techniques like heatmaps, decision trees, or saliency maps. This method can be used to find patterns in the data and reveal how the model interprets information.
Model Specific Models
Model-specific approaches are designed for particular machine-learning architectures or algorithms. These methods consist of:
- Visualizing the decision tree structure of the model will provide you an understanding of how the model makes decisions.
- Relevance across Layers Analyzing the role played by specific neurons or network layers to understand how deep neural networks make decisions.
- Techniques based on gradients: Analyzing the model's gradients in relation to the input characteristics to determine which features are most crucial for the choice.
- Sensitivity analysis is the process of examining how a model's output varies in response to changes in its input properties in order to determine which features are most crucial for the choice.
- In general, the choice of post-hoc explanation method will depend on the particular needs of the stakeholders and the features of the employed machine learning model.
Deep Learning Models
- MLP: Methods like sensitivity analysis and visualization can be utilized to comprehend the decision-making processes of MLPs.
- CNN: Methods like visualization and layer-wise relevance propagation can be used to explain the choices made by CNNs.
- RNN: Methods like visualization and gradient-based methods can be used to explain the choices made by RNNs.
- Overall, the decision about the post-hoc explanation technique will be based on the particular needs of the stakeholders as well as the features of the underlying machine learning model, including its complexity and design.
Model Dependent Techniques
- Using random forests and groups of classifiers, we may explain the model by highlighting the Significance of various features model simplification, local approaches, visuals, and examples are used to explain STEL, SWAF, and DeepSHAP.
- SVM: Extracting the Hyper-rectangle Rule.
- Neural networks: explanations by condensing the model, ranking features, and creating visualizations PatternNet, Pattern Attribution, Saliency Map, Grad-CAM, LRP, DGN, DeepRED, Interpretable Mimic Learning, DeepLIFT, Guided BackProp, Integrated Gradients (IG), DeConvNet, Grad-CAM, LRP, and DeepRED
Types of post-hoc predictions explanations
Post-hoc predictions are explanations of how a machine learning model came to a specific conclusion or prediction after the fact. Post-hoc predictions come in a variety of forms, including:
- Model-Agnostic approaches: Regardless of the architecture or underlying method, model-agnostic approaches can be used with any machine learning model. These methods consist of:
- Finding the features or input variables that had the biggest influence on the model's conclusion is known as determining the feature's importance.
- Local explanations: Justifying a model's choices made in relation to a particular instance or input.
- Providing an overarching explanation of how the model makes decisions based on all inputs.
- Provide information on how altering the input attributes may affect the model's judgment via counterfactual explanations.
- Visualization: Enabling a more intuitive comprehension of the decisions the model is making.
- Text explanations: These are written explanations that explain how a machine learning model came to a specific prediction, such as those that are presented in reports or emails.
- Examples-based explanations: These are explanations that show how a machine learning model arrived at a specific prediction using examples.
- Clarifications through model simplification: These are explanations that reduce a machine learning model's complexity so that non-experts can understand it.
Categorization of post-hoc techniques
- Base Attribution quantifies how much each input feature contributes to the final product of the model: Feature permutation, DeepLIFT, Saliency Maps, Guided GradCAM, LIME, and SHAP.
- Layer Attribution: Measures the contribution of neurons in a specific layer to the output of the model. Layer DeepLIFT, Internal Influence, Layer Activation, GradCAM, and Layer Integrated Gradients.
- Neuron Attribution: Measures the influence of each input feature on the activation of a particular hidden neuron: Neuron conductance, neuro gradient, integrated gradients in neurons, and deep lift neurons.
Post Hoc Explainability Analysis Example
Let's say a healthcare organization has created a machine learning model to forecast the likelihood that a patient with a chronic ailment, like heart disease, will need to be readmitted to the hospital. The model considers a number of variables, including age, gender, comorbidities, medication, and test findings.
The healthcare provider wants to guarantee that the model is precise, trustworthy, and simple for patients and professionals to understand. The healthcare provider can do this by examining the model's decision-making process using post-hoc explainability tools. The business, for instance, could employ the following methods:
1. Feature Importance: The healthcare organization can use permutation importance to pinpoint the elements that the model considers to be most crucial. Let's say the healthcare organization determines that the key components of the model are comorbidities, test findings, and medicines.
2. Local Explanations: To explain the choice made by the model for a particular patient, the healthcare provider can use LIME (Local Interpretable Model-Agnostic Explanations). Let's say the healthcare provider receives a patient with a high risk of being readmitted to the hospital based on the model. LIME can explain to the practitioner why the model indicated a high risk for the patient and assist the healthcare organization in understanding this.
3. SHAP (Shapley Additive Explanations) can be used by the healthcare industry to comprehend the total contribution of each feature to the model's decision-making process. Let's say the healthcare organization discovers that the model's ability to make decisions is most strongly influenced by test results.
4. The healthcare provider can create counterfactual explanations using CEM (Counterfactual Explanations via Model Extraction) and learn how altering the input attributes can affect the model's conclusion. Let's say the healthcare provider wants to know how the model would forecast the likelihood of a patient needing to return to the hospital while using a different drug. The healthcare provider can use CEM to create a hypothetical counterexample and comprehend how the model would evaluate the patient's risk.
5. Visualization: To provide customers a more intuitive grasp of how the model makes decisions, the healthcare organization can employ visualization approaches like heatmaps or decision trees. The healthcare organization can use these strategies to find patterns in the data and gain an understanding of how the model interprets information.
The healthcare provider can make sure that its machine learning model is precise, dependable, and simple for physicians and patients to understand by applying post-hoc explainability tools. This can enhance patient outcomes and guarantee that the healthcare provider is using the model's predictions to make ethical decisions.
Key points to remember
The following are important things to keep in mind while using XAI's post-hoc explainability techniques:
- Following training, machine learning models' decision-making processes can be explained using post-hoc approaches.
- Feature importance, local explanations, global explanations, counterfactual explanations, and visualization are just a few of the post-hoc methods that are available.
- Depending on the particular application and the user's needs, the best technique will be used.
- Although post-hoc methods can shed light on the information processing and decision-making processes of models, they cannot ensure that models are impartial, fair, or moral.
- Explainability is not always feasible or desired, especially when models are very complex or when privacy issues need to be taken into account.
- To assure the fairness, correctness, and dependability of AI systems, post-hoc explainability approaches should be used in concert with other techniques, such as model validation and testing.
- Post-hoc explainability approaches are a crucial first step in making AI systems more transparent and trustworthy because they can assist promote trust, accountability, and transparency in AI systems.
Conclusion
Techniques for post-hoc explainability are crucial for comprehending how machine learning models make decisions. These methods can aid in raising the level of trust, transparency, and accountability in AI systems by revealing how a model is thinking through information and making judgments.
A number of post-hoc explainability methods, such as feature importance, local explanations, global explanations, counterfactual explanations, and visualization, are frequently employed in the field of XAI. The best strategy to utilize will depend on the particular application and needs of the user. Each of these strategies has advantages and disadvantages of its own. It's crucial to remember that while post-hoc procedures can offer insightful information, they cannot ensure that a model is impartial, ethical, or fair. It's also critical to recognize that explainability isn't always feasible or desired, particularly when models are overly complicated or privacy issues must be taken into account.
Post-hoc explainability approaches, in general, are a crucial step toward developing more transparent and reliable AI systems, but they should be used in concert with other techniques, like model validation and testing, to guarantee the fairness, correctness, and dependability of AI systems.
References
