xai4tsc.evaluation.base
=======================

.. py:module:: xai4tsc.evaluation.base

.. autoapi-nested-parse::

   Evaluator base classes: ``EvaluatorBase`` ABC and ``QuantusEvaluator``.



Attributes
----------

.. autoapisummary::

   xai4tsc.evaluation.base.logger


Classes
-------

.. autoapisummary::

   xai4tsc.evaluation.base.EvaluatorBase
   xai4tsc.evaluation.base.QuantusEvaluator


Module Contents
---------------

.. py:data:: logger

.. py:class:: EvaluatorBase

   Bases: :py:obj:`abc.ABC`


   Base class for all XAI evaluation metrics.

   Subclass this and implement :meth:`evaluate`, then register the subclass with
   :func:`xai4tsc.register_metric` to make it available by name. The registry value
   is a *callable producing an evaluator*, so a subclass (instantiated with its
   constructor params) satisfies the contract directly.

   Example::

       class MyEvaluator(EvaluatorBase):
           metric_name = "my_metric"

           def evaluate(self, model, explanation, data, labels, device="cpu", **kw):
               return float(np.mean(np.abs(explanation.exp_values)))

       xai4tsc.register_metric("MyMetric", MyEvaluator)

   Domain applicability
   --------------------
   Like explainers, evaluators declare where they apply via two Quantus-style class
   attributes: :attr:`data_applicability` (a set of ``DataType`` members) and
   :attr:`required_domains` (a set of ``Domain`` members the explanation must be in).
   The runner skips a metric whose ``required_domains`` does not contain the
   explanation's domain; an empty set means "applies to any domain" (the Quantus
   default).

   Contract — attributions are real-valued
   ---------------------------------------
   Metric backends operate on **real** attributions: Quantus preprocessing and the
   perturbation metrics' ``np.argsort`` are undefined on complex arrays. Frequency /
   time-frequency explanations may be **complex** (the explanation-space wrappers
   transform the relevance itself), while others in the same domain are real
   (FreqRISE, random baselines). Backend evaluators must therefore pass attributions
   through :meth:`real_attribution` before handing them to the underlying metric.


   .. py:attribute:: metric_name
      :type:  str
      :value: ''


      Human-readable name of the metric.


   .. py:attribute:: data_applicability
      :type:  ClassVar[set[xai4tsc.xai._types.DataType]]

      Data domains this evaluator applies to (Quantus-style; defaults to agnostic).


   .. py:attribute:: required_domains
      :type:  ClassVar[set[xai4tsc.xai._types.Domain]]

      Explanation domains this evaluator requires; empty means any domain.


   .. py:method:: real_attribution(a_batch: numpy.ndarray | None) -> numpy.ndarray | None
      :staticmethod:


      Reduce a complex attribution to its magnitude; pass real arrays through.

      Frequency / time-frequency explanations may be complex; metric backends
      operate on real values, so complex attributions are reduced to magnitude
      (``np.abs``). Real attributions are returned unchanged so the metric's own
      ``abs`` handling still governs them.

      :param a_batch: The attribution array (possibly complex), or ``None``.
      :type a_batch: np.ndarray or None

      :returns: A real-valued attribution, or ``None`` if *a_batch* was ``None``.
      :rtype: np.ndarray or None



   .. py:method:: evaluate(model: torch.nn.Module, explanation: xai4tsc.xai._types.Explanation, data: numpy.ndarray, labels: numpy.ndarray, device: str = 'cpu', **kwargs: object) -> float | numpy.ndarray
      :abstractmethod:


      Run the evaluation metric.

      :param model: The PyTorch model being explained (an :class:`~torch.nn.Module`).
      :param explanation: :class:`~xai4tsc.xai.Explanation` dataclass containing
                          ``exp_values``, ``data``, ``labels``, and ``encoder``.
      :param data: Input samples to evaluate on.
      :type data: np.ndarray
      :param labels: Ground-truth labels.
      :type labels: np.ndarray
      :param device: Compute device.
      :type device: str
      :param \*\*kwargs: Additional metric-specific options.

      :returns: Metric score(s).
      :rtype: float or np.ndarray



.. py:class:: QuantusEvaluator(name: str, **metric_class_params: object)

   Bases: :py:obj:`EvaluatorBase`


   The single adapter for every `Quantus`_ metric.

   .. _Quantus: https://github.com/understandingai/Quantus

   Initialized with a metric **name** (a key of the ``QUANTUS_METRICS`` lookup table),
   not a class — so the registry can map every Quantus metric in via one class, e.g.
   ``partial(QuantusEvaluator, name)``.

   Example::

       evaluator = QuantusEvaluator(
           "Faithfulness Correlation",
           normalise=True,
           abs=True,
           disable_warnings=True,
       )
       score = evaluator.evaluate(model, explanation, data, labels)



   .. py:attribute:: metric_name

      Human-readable name of the metric.


   .. py:method:: evaluate(model: torch.nn.Module, explanation: xai4tsc.xai._types.Explanation, data: numpy.ndarray, labels: numpy.ndarray, device: str = 'cpu', **call_params: object) -> float | numpy.ndarray

      Instantiate the Quantus metric and run it.

      The metric is reinstantiated on each call to avoid state leakage
      between evaluations.