Seminar explainable AI (XAI-Seminar)

Mini Student’s Glossary (incomplete, but hopefully helpful for the newcomer student):

Anchors = developed by Ribeiro et al. 2018, works similar as LIME but use rules to define explanations by local regions.

Ante-hoc Explainability (AHE) = such models are interpretable by design, e.g. glass-box approaches; typical examples include linear regression, decision trees/lists, random forests, Naive Bayes and fuzzy inference systems; or GAMs, Stochastic AOGs, and deep symbolic networks; they have a long tradition and can be designed from expert knowledge or from data and are useful as framework for the interaction between human knowledge and hidden knowledge in the data.

BETA = Black Box Explanation through Transparent Approximation, developed by Lakkarju, Bach & Leskovec (2016) it learns two-level decision sets, where each rule explains the model behaviour; this is an increasing problem in daily use of AI/ML, see e.g. http://news.mit.edu/2019/better-fact-checking-fake-news-1017

Bias = inability for a ML method to represent the true relationship; High bias can cause an algorithm to miss the relevant relations between features and target outputs (underfitting);

Causability = is a property of a human (natural intelligence) and a measurement for the degree of human understanding; we have developed the System Causability Scale (SCS). 

Counterfactual = a hypothesis that is contrary to the facts (similar to counterexample), or a hypothetical state of the world, used to assess the impact of an action in the real-world, or a conditional statement in which the conditional clause is false, as “what-if” – this is very important for future human-ai interfaces

Counterexample = an exception of a proposed general rule or law and appears as an example which disproves a general statement made.

Decomposition = process of resolving relationships into the consituent components (hopefully representing the relevant interest). Highly theoretical, because in real-world this is hard due to the complexity (e.g. noise) and untraceable imponderabilities on our observations.

Deduction = deriving of a conclusion by reasoning

Explainability = motivated by the opaqueness of so called “black-box” approaches it is the ability to provide an explanation on why a machine decision has been reached (e.g. why is it a cat what the deep network recognized). Finding an appropriate explanation is difficult, because this needs understanding the context and providing a description of causality and consequences of a given fact. (German: Erklärbarkeit; siehe auch: Verstehbarkeit, Nachvollziehbarkeit, Zurückverfolgbarkeit, Transparenz)

Explanation = set of statements to describe a given set of facts to clarify causality, context and consequences thereof and is a core topic of knowledge discovery involving “why” questionss (“Why is this a cat?”). (German: Erklärung, Begründung)

Explanatory power = is the ability of a set hypothesis to effectively explain the subject matter it pertains to (opposite: explanatory impotence).

Explicit Knowledge = you can easy explain it by articulating it via natural language etc. and share it with others.

European General Data Protection Regulation (EU GDPR) =  Regulation EU 2016/679 – see the EUR-Lex 32016R0679 , will make black-box approaches difficult to use, because they often are not able to explain why a decision has been made (see explainable AI).

Feature Space = the n-dimensions of the feature variables extracted by ML excluding the target function. The dimension n is determined by the number of feature parameters.

Gaussian Process (GP) = collection of stochastic variables indexed by time or space so that each of them constitute a multidimensional Gaussian distribution; provides a probabilistic approach to learning in kernel machines (See: Carl Edward Rasmussen & Christopher K.I. Williams 2006. Gaussian processes for machine learning, Cambridge (MA), MIT Press); this can be used for explanations. (see also: Visual Exploration Gaussian)

Gradient = a vector providing the direction of maximum rate of change.

Ground truth = generally information provided by direct observation (i.e. empirical evidence) instead of provided by inference. For us it is the gold standard, i.e. the ideal expected result (100 % true);

Interactive Machine Learning (iML) = machine learning algorithms which can interact with – partly human – agents and can optimize its learning behaviour trough this interaction. Holzinger, A. 2016. Interactive Machine Learning for Health Informatics: When do we need the human-in-the-loop? Brain Informatics (BRIN), 3, (2), 119-131.

Inverse Probability = an older term for the probability distribution of an unobserved variable, and was described by De Morgan 1837, in reference to Laplace’s (1774) method of probability.

Implicit Knowledge = very hard to articulate, we do it but cannot explain it (also tacit knowlege).

In-Vitro Diagnostic medical devices directive (IVDR) = with effect on May, 26, 2022 this will also need re-traceability and interpretability for AI-assisted medical applications [see: EUR LEX IVDR]   in-vitro = “in the glass”, studies performed with microorganisms, cells, or biological molecules outside their normal biological context; in-vivo = “within live” – as opposed to a tissue extract; in-silico = performed  via computer simulation; in-situ = “real-world” context.

KANDINSKY-Patterns = “a swiss knife for the study of explainability” – see https://www.youtube.com/watch?v=UuiV0icAlRs

Kernel = class of algorithms for pattern analysis e.g. support vector machine (SVM); very useful for explainable AI.

Kernel trick = transforming data into another dimension that has a clear dividing margin between the classes.

LORE = LOcal Rule-based Explanations by Pastor & Baralis, 2019, use a genetic algorithm to identify most important features.

Multi-Agent Systems (MAS) = include collections of several independent agents, could also be a mixture of computer agents and human agents. An exellent pointer of the later one is: Jennings, N. R., Moreau, L., Nicholson, D., Ramchurn, S. D., Roberts, S., Rodden, T. & Rogers, A. 2014. On human-agent collectives. Communications of the ACM, 80-88.

Post-hoc Explainability (PHE) = such models are designed for interpreting black-box models and provide local explanations for a specific decision and re-enact on request, typical examples include LIME, BETA, LRP, or Local Gradient Explanation Vectors, prediction decomposition or simply feature selection.

Preference learning (PL) = concerns problems in learning to rank, i.e. learning a predictive preference model from observed preference information, e.g. with label ranking, instance ranking, or object ranking.  Fürnkranz, J., Hüllermeier, E., Cheng, W. & Park, S.-H. 2012. Preference-based reinforcement learning: a formal framework and a policy iteration algorithm. Machine Learning, 89, (1-2), 123-156.

Saliency map = image showing in a different representation (usually easier for human perception) each pixel’s quality.

Tacit Knowledge = Knowledge gained from personal experience that is even more difficult to express than implicite knowledge.

TCAV = Testing with Concept Activation Vectors developed by the Google Group around Kim et al. (2017) automatically creates a list of human understandable concepts (concept learning).

Transfer Learning (TL) = The ability of an algorithm to recognize and apply knowledge and skills learned in previous tasks to novel tasks or new domains, which share some commonality. Central question: Given a target task, how do we identify the commonality between the task and previous tasks, and transfer the knowledge from the previous tasks to the target one?
Pan, S. J. & Yang, Q. 2010. A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering, 22, (10), 1345-1359, doi:10.1109/tkde.2009.191.

Zero-shot learning = we see this as an extreme case of contextual domain adaptation.