xai4tsc.evaluation.base

Evaluator base classes: EvaluatorBase ABC and QuantusEvaluator.

Attributes

logger

Classes

EvaluatorBase	Base class for all XAI evaluation metrics.
QuantusEvaluator	The single adapter for every Quantus metric.

Module Contents

xai4tsc.evaluation.base.logger

class xai4tsc.evaluation.base.EvaluatorBase

Bases: abc.ABC

Base class for all XAI evaluation metrics.

Subclass this and implement evaluate(), then register the subclass with xai4tsc.register_metric() to make it available by name. The registry value is a callable producing an evaluator, so a subclass (instantiated with its constructor params) satisfies the contract directly.

Example:

class MyEvaluator(EvaluatorBase):
    metric_name = "my_metric"

    def evaluate(self, model, explanation, data, labels, device="cpu", **kw):
        return float(np.mean(np.abs(explanation.exp_values)))

xai4tsc.register_metric("MyMetric", MyEvaluator)

Domain applicability

Like explainers, evaluators declare where they apply via two Quantus-style class attributes: data_applicability (a set of DataType members) and required_domains (a set of Domain members the explanation must be in). The runner skips a metric whose required_domains does not contain the explanation’s domain; an empty set means “applies to any domain” (the Quantus default).

Contract — attributions are real-valued

Metric backends operate on real attributions: Quantus preprocessing and the perturbation metrics’ np.argsort are undefined on complex arrays. Frequency / time-frequency explanations may be complex (the explanation-space wrappers transform the relevance itself), while others in the same domain are real (FreqRISE, random baselines). Backend evaluators must therefore pass attributions through real_attribution() before handing them to the underlying metric.

metric_name: str = ''

Human-readable name of the metric.

data_applicability: ClassVar[set[xai4tsc.xai._types.DataType]]

Data domains this evaluator applies to (Quantus-style; defaults to agnostic).

required_domains: ClassVar[set[xai4tsc.xai._types.Domain]]

Explanation domains this evaluator requires; empty means any domain.

static real_attribution(a_batch: numpy.ndarray | None) → numpy.ndarray | None

Reduce a complex attribution to its magnitude; pass real arrays through.

Frequency / time-frequency explanations may be complex; metric backends operate on real values, so complex attributions are reduced to magnitude (np.abs). Real attributions are returned unchanged so the metric’s own abs handling still governs them.

Parameters:

a_batch (np.ndarray or None) – The attribution array (possibly complex), or None.

Returns:

A real-valued attribution, or None if a_batch was None.

Return type:

np.ndarray or None

abstractmethod evaluate(model: torch.nn.Module, explanation: xai4tsc.xai._types.Explanation, data: numpy.ndarray, labels: numpy.ndarray, device: str = 'cpu', **kwargs: object) → float | numpy.ndarray

Run the evaluation metric.

Parameters:

• model – The PyTorch model being explained (an Module).

• explanation – Explanation dataclass containing exp_values, data, labels, and encoder.

• data (np.ndarray) – Input samples to evaluate on.

• labels (np.ndarray) – Ground-truth labels.

• device (str) – Compute device.

• **kwargs – Additional metric-specific options.

Returns:

Metric score(s).

Return type:

float or np.ndarray

class xai4tsc.evaluation.base.QuantusEvaluator(name: str, **metric_class_params: object)

Bases: EvaluatorBase

The single adapter for every Quantus metric.

Initialized with a metric name (a key of the QUANTUS_METRICS lookup table), not a class — so the registry can map every Quantus metric in via one class, e.g. partial(QuantusEvaluator, name).

Example:

evaluator = QuantusEvaluator(
    "Faithfulness Correlation",
    normalise=True,
    abs=True,
    disable_warnings=True,
)
score = evaluator.evaluate(model, explanation, data, labels)

metric_name

Human-readable name of the metric.

evaluate(model: torch.nn.Module, explanation: xai4tsc.xai._types.Explanation, data: numpy.ndarray, labels: numpy.ndarray, device: str = 'cpu', **call_params: object) → float | numpy.ndarray

Instantiate the Quantus metric and run it.

The metric is reinstantiated on each call to avoid state leakage between evaluations.