xai4tsc.evaluation.timefrequency_auc

Time-frequency localization AUC metric.

Scores how well a frequency / time-frequency attribution localizes onto the ground-truth discriminative regions of a sample. For each sample a boolean ground-truth mask is built in the explanation’s coefficient grid from the dataset metadata (the {class: [regions]} produced by e.g. FreqShapesDataset), and the ROC-AUC of the (real) attribution against that mask is computed: 1.0 means the attribution ranks every discriminative coefficient above every non-discriminative one, 0.5 is chance.

Unlike the perturbation metrics this is a ground-truth metric — it needs the per-sample regions, which TimeFrequencyAUCEvaluator.evaluate() reads as metadata directly off the explanation. It is the synthetic-data counterpart to the frequency / time-frequency perturbation metrics.

Attributes

logger

Classes

TimeFrequencyAUCEvaluator

Ground-truth localization AUC for frequency / time-frequency attributions.

Module Contents

xai4tsc.evaluation.timefrequency_auc.logger
class xai4tsc.evaluation.timefrequency_auc.TimeFrequencyAUCEvaluator(transform: xai4tsc.utils.fourier_transforms.DomainTransform | None = None, metadata: list | None = None, return_aggregate: bool = True, abs_attribution: bool = True)

Bases: xai4tsc.evaluation.base.EvaluatorBase

Ground-truth localization AUC for frequency / time-frequency attributions.

For each sample, the ROC-AUC of the (real) attribution against the discriminative ground-truth mask is computed; samples with no discriminative regions (or a fully-masked grid) are skipped (AUC is undefined there).

Parameters:

  • transform (DomainTransform, optional) – The explanation’s transform; supplies the STFT grid geometry (n_fft / hop_length). Read from the explanation when left None.

  • metadata (list, optional) – Per-sample dataset metadata aligned to the attribution rows. Read from Explanation.metadata when left None; required — without it (on either) the metric returns nan and warns.

  • return_aggregate (bool) – If True (default) return the mean AUC over valid samples; otherwise return the per-sample AUC list (nan for undefined samples).

  • abs_attribution (bool) – Rank by attribution magnitude (default True); set False to rank by signed value.

data_applicability: ClassVar[set[xai4tsc.xai._types.DataType]]

Data domains this evaluator applies to (Quantus-style; defaults to agnostic).

required_domains: ClassVar[set[xai4tsc.xai._types.Domain]]

Explanation domains this evaluator requires; empty means any domain.

transform = None
metadata = None
return_aggregate = True
abs_attribution = True
evaluate(model: torch.nn.Module, explanation: xai4tsc.xai._types.Explanation, data: numpy.ndarray, labels: numpy.ndarray, device: str = 'cpu', **kwargs: object) → float | list

Compute the localization AUC for the batch.

The transform and per-sample ground-truth metadata are read from the constructor if set, else from the explanation.

Parameters:

  • model – Unused (this is a ground-truth metric, not a perturbation one).

  • explanation (Explanation) – Provides the attribution (exp_values), the domain transform, and the per-sample ground-truth metadata.

  • data (np.ndarray) – Time-domain inputs (B, C, T) — sets the frequency scale T.

  • labels (np.ndarray) – Target class per sample (unused; the mask unions all classes).

  • device (str) – Unused.

  • **kwargs (object) – Ignored.

Returns:

Mean AUC over valid samples (return_aggregate) or the per-sample AUC list. nan where the metric is undefined / inapplicable.

Return type:

float or list