signxai.tf_signxai.methods_impl.innvestigate.analyzer package

Subpackages

Submodules

signxai.tf_signxai.methods_impl.innvestigate.analyzer.base module

exception signxai.tf_signxai.methods_impl.innvestigate.analyzer.base.NotAnalyzeableModelException[source]

Bases: Exception

Indicates that the model cannot be analyzed by an analyzer.

class signxai.tf_signxai.methods_impl.innvestigate.analyzer.base.AnalyzerBase(model, disable_model_checks=False, **kwargs)[source]

Bases: object

The basic interface of an iNNvestigate analyzer.

This class defines the basic interface for analyzers:

# >>> model = create_keras_model() # >>> a = Analyzer(model) # >>> a.fit(X_train) # If analyzer needs training. # >>> analysis = a.analyze(X_test) # >>> # >>> state = a.save() # >>> a_new = A.load(*state) # >>> analysis = a_new.analyze(X_test)

Parameters:

  • model – A tf.keras model.

  • disable_model_checks – Do not execute model checks that enforce compatibility of analyzer and model.

Note

To develop a new analyzer derive from AnalyzerNetworkBase.

__init__(model, disable_model_checks=False, **kwargs)[source]
fit(*args, **kwargs)[source]

Stub that eats arguments. If an analyzer needs training include TrainerMixin.

Parameters:

disable_no_training_warning – Do not warn if this function is called despite no training is needed.

fit_generator(*args, **kwargs)[source]

Stub that eats arguments. If an analyzer needs training include TrainerMixin.

Parameters:

disable_no_training_warning – Do not warn if this function is called despite no training is needed.

analyze(X)[source]

Analyze the behavior of model on input X.

Parameters:

X – Input as expected by model.

class signxai.tf_signxai.methods_impl.innvestigate.analyzer.base.TrainerMixin[source]

Bases: object

Mixin for analyzer that adapt to data.

This convenience interface exposes a Keras like training routing to the user.

fit(X=None, batch_size=32, **kwargs)[source]

Takes the same parameters as Keras’s model.fit() function.

fit_generator(*args, **kwargs)[source]

Takes the same parameters as Keras’s model.fit_generator() function.

class signxai.tf_signxai.methods_impl.innvestigate.analyzer.base.OneEpochTrainerMixin[source]

Bases: TrainerMixin

Exposes the same interface and functionality as TrainerMixin except that the training is limited to one epoch.

fit(*args, **kwargs)[source]

Same interface as fit() of TrainerMixin except that the parameter epoch is fixed to 1.

fit_generator(*args, **kwargs)[source]

Same interface as fit_generator() of TrainerMixin except that the parameter epoch is fixed to 1.

class signxai.tf_signxai.methods_impl.innvestigate.analyzer.base.AnalyzerNetworkBase(model, allow_lambda_layers=False, **kwargs)[source]

Bases: AnalyzerBase

Convenience interface for analyzers.

This class provides helpful functionality to create analyzer’s. Basically it:

  • takes the input model and adds a layer that selects the desired output neuron to analyze.

  • passes the new model to _create_analysis() which should return the analysis as Keras tensors.

  • compiles the function and serves the output to analyze() calls.

  • allows _create_analysis() to return tensors that are intercept for debugging purposes.

Parameters:

allow_lambda_layers – Allow the model to contain lambda layers.

__init__(model, allow_lambda_layers=False, **kwargs)[source]
create_analyzer_model(**kwargs)[source]

Creates the analyze functionality. If not called beforehand it will be called by analyze().

analyze(X, neuron_selection='max_activation', explained_layer_names=None, stop_mapping_at_layers=None, r_init=None, f_init=None, **kwargs)[source]

Takes an array-like input X and explains it. Also applies postprocessing to the explanation

Parameters:

  • X – tensor or np.array of Input to be explained. Shape (n_ins, batch_size, …) in model has multiple inputs, or (batch_size, …) otherwise

  • neuron_selection – neuron_selection parameter. Used to only compute explanation w.r.t. specific output neurons. One of the following: - None or “all” - “max_activation” - int - list or np.array of int, with length equal to batch size

  • explained_layer_names – None or “all” or list of layer names whose explanations should be returned. Can be used to obtain intermediate explanations or explanations of multiple layers if layer names provided, a dictionary is returned

  • stop_mapping_at_layers – None or list of layers to stop mapping at (“output” layers)

  • r_init – None or Scalar or Array-Like or Dict {layer_name:scalar or array-like} reverse initialization value. Value with which the explanation is initialized.

  • f_init – None or Scalar or Array-Like or Dict {layer_name:scalar or array-like} forward initialization value. Value with which the forward is initialized.

Returns Dict of the form {layer name (string):

explanation (numpy.ndarray)}

get_explanations(explained_layer_names=None)[source]

Get results of (previously computed) explanation. explanation of layer i has shape equal to input_shape of layer i.

Parameters:

explained_layer_names – None or “all” or list of strings containing the names of the layers. if explained_layer_names == ‘all’, explanations of all layers are returned.

Returns Dict of the form {layer name (string):

explanation (numpy.ndarray)}

class signxai.tf_signxai.methods_impl.innvestigate.analyzer.base.ReverseAnalyzerBase(model, **kwargs)[source]

Bases: AnalyzerNetworkBase

Convenience class for analyzers that revert the model’s structure. This class contains many helper functions around the graph reverse function innvestigate.utils.keras.graph.reverse_model(). The deriving classes should specify how the graph should be reverted.

__init__(model, **kwargs)[source]

signxai.tf_signxai.methods_impl.innvestigate.analyzer.deeplift module

class signxai.tf_signxai.methods_impl.innvestigate.analyzer.deeplift.DeepLIFT(model, *args, **kwargs)[source]

Bases: ReverseAnalyzerBase

DeepLIFT-rescale algorithm

This class implements the DeepLIFT algorithm using the rescale rule (as in DeepExplain (Ancona et.al.)).

WARNING: This implementation contains bugs.

Parameters:

model – A Keras model.

__init__(model, *args, **kwargs)[source]
class signxai.tf_signxai.methods_impl.innvestigate.analyzer.deeplift.DeepLIFTWrapper(model, **kwargs)[source]

Bases: AnalyzerNetworkBase

Wrapper around DeepLIFT package

This class wraps the DeepLIFT package. For further explanation of the parameters check out: https://github.com/kundajelab/deeplift

Parameters:

  • model – A Keras model.

  • nonlinear_mode – The nonlinear mode parameter.

  • reference_inputs – The reference input used for DeepLIFT.

  • verbose – Verbosity of the DeepLIFT package.

Note:

Requires the deeplift package.

__init__(model, **kwargs)[source]
analyze(X, neuron_selection=None)[source]

Takes an array-like input X and explains it. Also applies postprocessing to the explanation

Parameters:

  • X – tensor or np.array of Input to be explained. Shape (n_ins, batch_size, …) in model has multiple inputs, or (batch_size, …) otherwise

  • neuron_selection – neuron_selection parameter. Used to only compute explanation w.r.t. specific output neurons. One of the following: - None or “all” - “max_activation” - int - list or np.array of int, with length equal to batch size

  • explained_layer_names – None or “all” or list of layer names whose explanations should be returned. Can be used to obtain intermediate explanations or explanations of multiple layers if layer names provided, a dictionary is returned

  • stop_mapping_at_layers – None or list of layers to stop mapping at (“output” layers)

  • r_init – None or Scalar or Array-Like or Dict {layer_name:scalar or array-like} reverse initialization value. Value with which the explanation is initialized.

  • f_init – None or Scalar or Array-Like or Dict {layer_name:scalar or array-like} forward initialization value. Value with which the forward is initialized.

Returns Dict of the form {layer name (string):

explanation (numpy.ndarray)}

signxai.tf_signxai.methods_impl.innvestigate.analyzer.deeptaylor module

signxai.tf_signxai.methods_impl.innvestigate.analyzer.gradient_based module

class signxai.tf_signxai.methods_impl.innvestigate.analyzer.gradient_based.Gradient(model, postprocess=None, **kwargs)[source]

Bases: ReverseAnalyzerBase

Gradient analyzer.

Returns as analysis the function value with respect to the input. The gradient is computed via the library’s network reverting.

Parameters:

model – A Keras model.

__init__(model, postprocess=None, **kwargs)[source]
class signxai.tf_signxai.methods_impl.innvestigate.analyzer.gradient_based.InputTimesGradient(model, **kwargs)[source]

Bases: Gradient

Input*Gradient analyzer.

Parameters:

model – A Keras model.

__init__(model, **kwargs)[source]
class signxai.tf_signxai.methods_impl.innvestigate.analyzer.gradient_based.Deconvnet(model, **kwargs)[source]

Bases: ReverseAnalyzerBase

Deconvnet analyzer.

Applies the “deconvnet” algorithm to analyze the model.

Parameters:

model – A Keras model.

__init__(model, **kwargs)[source]
class signxai.tf_signxai.methods_impl.innvestigate.analyzer.gradient_based.GuidedBackprop(model, **kwargs)[source]

Bases: ReverseAnalyzerBase

Guided backprop analyzer.

Applies the “guided backprop” algorithm to analyze the model.

Parameters:

model – A Keras model.

__init__(model, **kwargs)[source]
class signxai.tf_signxai.methods_impl.innvestigate.analyzer.gradient_based.IntegratedGradients(model, steps=16, **kwargs)[source]

Bases: PathIntegrator

Integrated gradient analyzer.

Applies the “integrated gradient” algorithm to analyze the model.

Parameters:

  • model – A Keras model.

  • steps – Number of steps to use average along integration path.

__init__(model, steps=16, **kwargs)[source]
class signxai.tf_signxai.methods_impl.innvestigate.analyzer.gradient_based.SmoothGrad(model, augment_by_n=16, **kwargs)[source]

Bases: GaussianSmoother

Smooth grad analyzer.

Applies the “smooth grad” algorithm to analyze the model.

Parameters:

  • model – A Keras model.

  • augment_by_n – Number of distortions to average for smoothing.

__init__(model, augment_by_n=16, **kwargs)[source]
class signxai.tf_signxai.methods_impl.innvestigate.analyzer.gradient_based.VarGrad(model, augment_by_n=16, **kwargs)[source]

Bases: VariationalSmoother

Var grad analyzer.

Applies the “var grad” algorithm to analyze the model.

Parameters:

  • model – A Keras model.

  • augment_by_n – Number of distortions to average for smoothing.

__init__(model, augment_by_n=16, **kwargs)[source]

signxai.tf_signxai.methods_impl.innvestigate.analyzer.misc module

class signxai.tf_signxai.methods_impl.innvestigate.analyzer.misc.Random(model, stddev=1, **kwargs)[source]

Bases: AnalyzerNetworkBase

Returns noise.

Returns the Gaussian noise as analysis.

Parameters:

  • model – A Keras model.

  • stddev – The standard deviation of the noise.

__init__(model, stddev=1, **kwargs)[source]
class signxai.tf_signxai.methods_impl.innvestigate.analyzer.misc.Input(model, allow_lambda_layers=False, **kwargs)[source]

Bases: AnalyzerNetworkBase

Returns the input.

Returns the input as analysis.

Parameters:

model – A Keras model.

signxai.tf_signxai.methods_impl.innvestigate.analyzer.pattern_based module

signxai.tf_signxai.methods_impl.innvestigate.analyzer.reverse_map module

class signxai.tf_signxai.methods_impl.innvestigate.analyzer.reverse_map.ReplacementLayer(layer, layer_next=[], r_init_constant=None, f_init_constant=None, **kwargs)[source]

Bases: object

Base class for providing explainability functionality.

This class wraps a single network layer, providing hooks for applying the layer and retrieving an explanation. Basically: * Any forward passes required for computing the explanation are defined in apply method. During the forward pass, a callback is given to all child layers to retrieve their explanations * Wrappers (e.g., a GradientTape) around the forward pass(es) that are required to compute an explanation can be defined and returned by wrap_hook method * In wrap_hook method, forward pass(es) are applied and Tapes defined * Explanation is computed in explain_hook method and then passed to callback functions of parent ReplacementLayers

Parameters:

  • layer – Layer (of base class tensorflow.keras.layers.Layer) of to wrap

  • layer_next – List of Layers in the network that receive output of layer (=child layers)

  • r_init_constant – If not None, defines a constant output mapping

  • f_init_constant – If not None, defines a constant activation mapping

This is just a base class. To extend this for specific XAI methods, - wrap_hook() - explain_hook() should be overwritten accordingly

__init__(layer, layer_next=[], r_init_constant=None, f_init_constant=None, **kwargs)[source]
try_explain(reversed_outs)[source]

callback function called by child layers when their explanation is computed.

  • aggregates explanations of all children

  • calls explain_hook to compute own explanation

  • sends own explanation to all parent layers by calling their callback functions

Parameters:

reversed_outs – the child layer’s explanation. None if this is the last layer.

try_apply(ins, callback=None, neuron_selection=None, stop_mapping_at_layers=None, r_init=None, f_init=None)[source]

Tries to apply own forward pass: * Aggregates inputs and callbacks of all parent layers * Performs a canonization of the neuron_selection parameter * Calls wrap_hook (wrapped forward pass(es)) * Calls _forward (forward result of forward pass to child layers)

:param ins output of own forward pass :param neuron_selection: neuron_selection parameter. One of the following:

  • None or “all”

  • “max_activation”

  • int

  • list or np.array of int, with length equal to batch size

Parameters:

stop_mapping_at_layers – None or list of layers to stop mapping at

:param callback callback function of the parent layer that called self.try_apply :param r_init: None or Scalar or Array-Like or Dict {layer_name:scalar or array-like} reverse initialization value. Value with with explanation is initialized (i.e., head_mapping). :param f_init: None or Scalar or Array-Like or Dict {layer_name:scalar or array-like} forward initialization value. Value with which the forward is initialized.

wrap_hook(ins, neuron_selection, stop_mapping_at_layers, r_init)[source]

hook that wraps and applies the layer function. E.g., by defining a GradientTape * should contain a call to self._neuron_select. * may define any wrappers around

Parameters:

  • ins – input(s) of this layer

  • neuron_selection – neuron_selection parameter (see try_apply)

  • stop_mapping_at_layers – None or stop_mapping_at_layers parameter (see try_apply)

  • r_init – reverse initialization value. Value with with explanation is initialized (i.e., head_mapping).

:returns output of layer function + any wrappers that were defined and are needed in explain_hook

To be extended for specific XAI methods

explain_hook(ins, reversed_outs, args)[source]

hook that computes the explanations. * Core XAI functionality

Parameters:

  • ins – input(s) of this layer

  • args – outputs of wrap_hook (any parameters that may be needed to compute explanation)

  • reversed_outs – either backpropagated explanation(s) of child layers, or None if this is the last layer

:returns explanation, or tensor of multiple explanations if the layer has multiple inputs (one for each)

To be extended for specific XAI methods

class signxai.tf_signxai.methods_impl.innvestigate.analyzer.reverse_map.GradientReplacementLayer(*args, **kwargs)[source]

Bases: ReplacementLayer

Simple extension of ReplacementLayer * Explains by computing gradients of outputs w.r.t. inputs of layer

__init__(*args, **kwargs)[source]
wrap_hook(ins, neuron_selection, stop_mapping_at_layers, r_init)[source]

hook that wraps and applies the layer function. E.g., by defining a GradientTape * should contain a call to self._neuron_select. * may define any wrappers around

Parameters:

  • ins – input(s) of this layer

  • neuron_selection – neuron_selection parameter (see try_apply)

  • stop_mapping_at_layers – None or stop_mapping_at_layers parameter (see try_apply)

  • r_init – reverse initialization value. Value with with explanation is initialized (i.e., head_mapping).

:returns output of layer function + any wrappers that were defined and are needed in explain_hook

To be extended for specific XAI methods

explain_hook(ins, reversed_outs, args)[source]

hook that computes the explanations. * Core XAI functionality

Parameters:

  • ins – input(s) of this layer

  • args – outputs of wrap_hook (any parameters that may be needed to compute explanation)

  • reversed_outs – either backpropagated explanation(s) of child layers, or None if this is the last layer

:returns explanation, or tensor of multiple explanations if the layer has multiple inputs (one for each)

To be extended for specific XAI methods

class signxai.tf_signxai.methods_impl.innvestigate.analyzer.reverse_map.ReverseModel(model, reverse_mappings, default_reverse_mapping, **kwargs)[source]

Bases: object

Defines a ReverseModel

ReverseModels are built from ReplacementLayer subclasses. A ReverseModel is defined via a list of Input ReplacementLayers (the input layers of the model)

and ReplacementLayers (the whole model)

Offers methods to

  • build

  • apply

  • get precomputed explanations from

  • save

  • load

the ReverseModel

__init__(model, reverse_mappings, default_reverse_mapping, **kwargs)[source]
build(model, reverse_mappings, default_reverse_mapping, **kwargs)[source]

Builds the ReverseModel by wrapping keras network layer(s) into ReplacementLayer(s)

Parameters:

  • model – tf.keras model to be replaced

  • reverse_mappings – mapping layer->reverse mapping (ReplacementLayer or some subclass thereof)

  • default_reverse_mapping – ReplacementLayer or some subclass thereof; default mapping to use

:returns -

apply(Xs, neuron_selection='max_activation', explained_layer_names=None, stop_mapping_at_layers=None, r_init=None, f_init=None)[source]

Computes an explanation by applying the ReverseModel

Parameters:

  • Xs – tensor or np.array of Input to be explained. Shape (n_ins, batch_size, …) in model has multiple inputs, or (batch_size, …) otherwise

  • neuron_selection – neuron_selection parameter. Used to only compute explanation w.r.t. specific output neurons. One of the following: - None or “all” - “max_activation” - int - list or np.array of int, with length equal to batch size

  • explained_layer_names – None or “all” or list of layer names whose explanations should be returned. Can be used to obtain intermediate explanations or explanations of multiple layers

  • stop_mapping_at_layers – None or list of layers to stop mapping at (“output” layers)

  • r_init – None or Scalar or Array-Like or Dict {layer_name:scalar or array-like} reverse initialization value. Value with which the explanation is initialized.

  • f_init – None or Scalar or Array-Like or Dict {layer_name:scalar or array-like} forward initialization value. Value with which the forward is initialized.

Returns Dict of the form {layer name (string):

explanation (numpy.ndarray)}

get_explanations(explained_layer_names=None)[source]

Get results of (previously computed) explanation. explanation of layer i has shape equal to input_shape of layer i.

Parameters:

explained_layer_names – None or “all” or list of strings containing the names of the layers. if explained_layer_names == ‘all’, explanations of all layers are returned.

Returns Dict of the form {layer name (string):

explanation (numpy.ndarray)}

save()[source]
load()[source]

signxai.tf_signxai.methods_impl.innvestigate.analyzer.wrapper module

class signxai.tf_signxai.methods_impl.innvestigate.analyzer.wrapper.WrapperBase(subanalyzer, *args, **kwargs)[source]

Bases: AnalyzerBase

Interface for wrappers around analyzers

This class is the basic interface for wrappers around analyzers.

Parameters:

subanalyzer – The analyzer to be wrapped.

__init__(subanalyzer, *args, **kwargs)[source]
analyze(*args, **kwargs)[source]

Analyze the behavior of model on input X.

Parameters:

X – Input as expected by model.

class signxai.tf_signxai.methods_impl.innvestigate.analyzer.wrapper.AugmentReduceBase(subanalyzer, *args, **kwargs)[source]

Bases: WrapperBase

Interface for wrappers that augment the input and reduce the analysis.

This class is an interface for wrappers that: * augment the input to the analyzer by creating new samples. * reduce the returned analysis to match the initial input shapes.

Parameters:

  • subanalyzer – The analyzer to be wrapped.

  • augment_by_n – Number of samples to create.

__init__(subanalyzer, *args, **kwargs)[source]
analyze(X, *args, **kwargs)[source]

Analyze the behavior of model on input X.

Parameters:

X – Input as expected by model.

class signxai.tf_signxai.methods_impl.innvestigate.analyzer.wrapper.GaussianSmoother(subanalyzer, *args, **kwargs)[source]

Bases: AugmentReduceBase

Wrapper that adds noise to the input and averages over analyses

This wrapper creates new samples by adding Gaussian noise to the input. The final analysis is an average of the returned analyses.

Parameters:

  • subanalyzer – The analyzer to be wrapped.

  • noise_scale – The stddev of the applied noise.

  • augment_by_n – Number of samples to create.

__init__(subanalyzer, *args, **kwargs)[source]
class signxai.tf_signxai.methods_impl.innvestigate.analyzer.wrapper.PathIntegrator(subanalyzer, *args, **kwargs)[source]

Bases: AugmentReduceBase

Integrated the analysis along a path

This analyzer: * creates a path from input to reference image. * creates steps number of intermediate inputs and

crests an analysis for them.

  • sums the analyses and multiplies them with the input-reference_input.

This wrapper is used to implement Integrated Gradients. We refer to the paper for further information.

Parameters:

  • subanalyzer – The analyzer to be wrapped.

  • steps – Number of steps for integration.

  • reference_inputs – The reference input.

__init__(subanalyzer, *args, **kwargs)[source]
analyze(X, *args, **kwargs)[source]

Analyze the behavior of model on input X.

Parameters:

X – Input as expected by model.

Module contents

signxai.tf_signxai.methods_impl.innvestigate.analyzer.create_analyzer(name, model, **kwargs)[source]

Instantiates the analyzer with the name ‘name’

This convenience function takes an analyzer name creates the respective analyzer.

Alternatively analyzers can be created directly by instantiating the respective classes.

Parameters:

  • name – Name of the analyzer.

  • model – The model to analyze, passed to the analyzer’s __init__.

  • kwargs – Additional parameters for the analyzer’s .

Returns:

An instance of the chosen analyzer.

Raises:

KeyError – If there is no analyzer with the passed name.