xai4tsc.evaluation.quantus_groups ================================= .. py:module:: xai4tsc.evaluation.quantus_groups .. autoapi-nested-parse:: What each Quantus metric consumes, and the explanation domains that follow from it. Quantus metrics do not all need the same inputs. A metric that reads only the attribution measures a property of the attribution itself and is therefore domain-agnostic; one that perturbs the input and re-queries the model needs the explanation in the **model-native domain**, the domain the model's inputs live in (``Explanation.data_domain``), because that is the space it can perturb and predict in. For a model trained on raw series that is the time domain; for one trained on spectra it is the frequency domain, and these metrics apply there instead. Quantus itself draws no such distinction - every metric inherits one ``__call__`` that asserts the attribution and the input correspond - so it is recorded here. The four groups are disjoint and cover every key of ``QUANTUS_METRICS``: ======================= =========================== === =========================== group its own code reads n explanation domain ======================= =========================== === =========================== ``ATTRIBUTION_ONLY`` ``a_batch`` 3 any ``MODEL_PERTURBATION`` ``x_batch`` + ``model`` 16 model-native ``LOCALISATION`` ``x_batch`` + ``s_batch`` 6 model-native (see below) ``RE_EXPLAIN`` ``+ explain_func`` 12 model-native ======================= =========================== === =========================== Derived by walking each metric's own ``evaluate_batch`` / ``evaluate_instance`` / ``custom_*_preprocess`` bodies with :mod:`ast`, excluding the inherited ``Metric.__call__`` that every subclass re-declares with the full parameter list. ``sandbox/fixtures/group_metrics.py`` re-derives the table from the installed ``quantus`` so it can be re-checked after a dependency bump. ``LOCALISATION`` is model-native for the time being because it is blocked on ``s_batch`` (ground-truth masks), which nothing supplies yet - not because of the domain. Once masks are derived, those six become domain-agnostic too: :class:`~xai4tsc.evaluation.timefrequency_auc.TimeFrequencyAUCEvaluator` already builds ground-truth masks on the coefficient grid. Attributes ---------- .. autoapisummary:: xai4tsc.evaluation.quantus_groups.QUANTUS_GROUPS xai4tsc.evaluation.quantus_groups.BLOCKED_BY_BACKEND Classes ------- .. autoapisummary:: xai4tsc.evaluation.quantus_groups.QuantusGroup Functions --------- .. autoapisummary:: xai4tsc.evaluation.quantus_groups.quantus_requires_native_domain xai4tsc.evaluation.quantus_groups.quantus_requires_explain_func xai4tsc.evaluation.quantus_groups.quantus_requires_s_batch Module Contents --------------- .. py:class:: QuantusGroup(*args, **kwds) Bases: :py:obj:`enum.Enum` Which inputs a Quantus metric's own code reads; see the module docstring. .. py:attribute:: ATTRIBUTION_ONLY :value: 'attribution-only' Reads the attribution alone, so it applies in any explanation domain. .. py:attribute:: MODEL_PERTURBATION :value: 'model-perturbation' Perturbs the input and re-queries the model. .. py:attribute:: LOCALISATION :value: 'localisation' Scores the attribution against a ground-truth mask (``s_batch``). .. py:attribute:: RE_EXPLAIN :value: 're-explain' Re-explains perturbed or randomised inputs via ``explain_func``. .. py:data:: QUANTUS_GROUPS :type: dict[str, QuantusGroup] Group of every metric in ``QUANTUS_METRICS``, keyed by the same display name. .. py:function:: quantus_requires_native_domain(name: str) -> bool Return whether the named Quantus metric needs the model-native domain. :param name: A key of ``QUANTUS_METRICS`` (the metric's display name). :type name: str :returns: ``True`` if the explanation must be in the same domain as the model's inputs, ``False`` if the metric applies in any domain. An unclassified name is treated as needing the native domain, the conservative choice. :rtype: bool .. py:function:: quantus_requires_explain_func(name: str) -> bool Return whether the named Quantus metric re-explains and so needs an explain func. :param name: A key of ``QUANTUS_METRICS`` (the metric's display name). :type name: str :returns: ``True`` for the ``RE_EXPLAIN`` group only. An unclassified name is treated as not requiring one: supplying the callable to a metric that ignores it is harmless, whereas demanding one that is never used would refuse a metric that works. :rtype: bool .. py:function:: quantus_requires_s_batch(name: str) -> bool Return whether the named Quantus metric scores against ground-truth masks. :param name: A key of ``QUANTUS_METRICS`` (the metric's display name). :type name: str :returns: ``True`` for the ``LOCALISATION`` group only. These metrics compare the attribution against the region a dataset declares as discriminative, so they are unusable without one — unlike the other groups, which need nothing but the model and the attribution. :rtype: bool .. py:data:: BLOCKED_BY_BACKEND :type: dict[str, str] Metrics whose Quantus implementation cannot serve a time series correctly. Four of them index a second spatial axis that a ``(B, C, T)`` series does not have - super-pixels, quadrants, 2-D padding, pixel neighbourhoods - and raise somewhere inside the library. Refusing them here turns an obscure ``IndexError`` or shape assertion into a stated reason, and keeps them out of the error column where they look like something worth fixing. ``Smooth MPRT`` is the dangerous one and the reason this table is not just about shapes: it *returns a number*. It calls the library's own explain function instead of the one it is given, so without this entry it would report a plausible score for an explainer it never ran.