xai4tsc.data.datasets ===================== .. py:module:: xai4tsc.data.datasets .. autoapi-nested-parse:: Dataset classes: ``UcrUeaDataset``, ``LocalDataset``, ``SyntheticDataset``. Attributes ---------- .. autoapisummary:: xai4tsc.data.datasets.logger xai4tsc.data.datasets.SYNTHETIC_DATASETS Classes ------- .. autoapisummary:: xai4tsc.data.datasets.UcrUeaDataset xai4tsc.data.datasets.LocalDataset xai4tsc.data.datasets.SyntheticDataset xai4tsc.data.datasets.FreqShapesDataset Functions --------- .. autoapisummary:: xai4tsc.data.datasets.register_synthetic_dataset xai4tsc.data.datasets.load_dataset Module Contents --------------- .. py:data:: logger .. py:class:: UcrUeaDataset(name: str, cache_dir: pathlib.Path | None = None, download: bool = True, pad_series: bool = False, max_samples: int | None = None, sample_strategy: str = 'random', max_series_length: int | None = None, series_position: str = 'first', use_predefined_splits: bool = False) Bases: :py:obj:`xai4tsc.data.base.DatasetBase` Download from the UCR/UEA archive and load raw (unsplit) data. .. py:attribute:: name Human-readable dataset identifier. .. py:attribute:: cache_dir .. py:attribute:: download :value: True .. py:attribute:: pad_series :value: False .. py:attribute:: use_predefined_splits :value: False .. py:method:: load() -> tuple Load raw data from the UCR/UEA archive via sktime. sktime downloads and caches the ``.ts`` files under ``cache_dir`` (passed as ``extract_path``). Restrictions (``max_samples``, ``max_series_length``) are applied after loading. The caller (``split()``) saves the processed train / val / test arrays to the split cache. .. py:method:: split(train_split: float = 0.8, val_split: float = 0.0, random_state: int = 42, encode: str = 'label', impute_missing: bool = False, rng: numpy.random.Generator | None = None, stratify: bool = True) -> tuple Load and split the dataset. When :attr:`use_predefined_splits` is ``True``, loads the official ``_TRAIN.ts`` and ``_TEST.ts`` files directly and ignores ``train_split``, ``val_split``, and ``stratify``. No validation set is produced. Otherwise delegates to :meth:`DatasetBase.split` for random / stratified splitting. .. py:class:: LocalDataset(path: str | pathlib.Path, name: str, data_format: str = 'numpy', pad_series: bool = False, max_samples: int | None = None, sample_strategy: str = 'random', max_series_length: int | None = None, series_position: str = 'first') Bases: :py:obj:`xai4tsc.data.base.DatasetBase` Load a single-file dataset from a local directory. The directory must contain one ``data*.npy`` file and one ``label*.json`` file (unsplit layout). For pre-split local data, use :meth:`~xai4tsc.data.DatasetBase.load_saved_splits` directly. .. py:attribute:: path .. py:attribute:: name Human-readable dataset identifier. .. py:attribute:: data_format :value: 'numpy' .. py:attribute:: pad_series :value: False .. py:method:: load() -> tuple Load raw data from the local directory using the configured format loader. :returns: ``(data, labels, metadata)`` with data in ``(B, C, T)`` layout. :rtype: tuple .. py:class:: SyntheticDataset(name: str, cache_dir: str | pathlib.Path | None = None, seed: int = 0, max_samples: int | None = None, sample_strategy: str = 'random', max_series_length: int | None = None, series_position: str = 'first') Bases: :py:obj:`xai4tsc.data.base.DatasetBase`, :py:obj:`abc.ABC` Base class for programmatically generated datasets. Subclasses implement :meth:`generate_dataset` (pure, deterministic given the constructor's ``seed`` + parameters). The concrete :meth:`load` here owns **caching**: it first looks for a previously generated dataset on disk and only regenerates on a miss. Caching layout (under ``cache_dir/synthetic//``): - ``data.npy`` — generated array ``(B, C, T)`` - ``labels.json`` — labels as records (``[{"label": ...}, ...]``; multi-hot labels are stored as lists) - ``metadata.json`` — per-sample metadata records (optional) **Metadata convention:** one row per sample, index-aligned with ``data`` in the same order. For ground-truth localization datasets it carries the discriminative regions keyed **per class** (``ground_truth``) plus distractors (``non_discriminative``); see the package data conventions. Example:: class GaussianDataset(SyntheticDataset): def _cache_params(self): return {"n": self.n_samples, "c": self.n_classes, "t": self.length} def generate_dataset(self): rng = np.random.default_rng(self.seed) data = rng.standard_normal((self.n_samples, 1, self.length)) labels = rng.integers(0, self.n_classes, self.n_samples) return data.astype("float32"), labels, None .. py:attribute:: name Human-readable dataset identifier. .. py:attribute:: cache_dir .. py:attribute:: seed :value: 0 .. py:method:: generate_dataset() -> tuple :abstractmethod: Generate and return ``(data, labels, metadata)`` — deterministic. ``data`` is ``(B, C, T)`` float32; ``labels`` is ``(B,)`` integer or ``(B, n_attrs)`` multi-hot (collapsed to class indices by the ``multihot`` encoder); ``metadata`` is the per-sample DataFrame (or ``None``). .. py:method:: cache_subdir(root: str | pathlib.Path) -> pathlib.Path Return this dataset's cache directory under ``root/synthetic/``. .. py:method:: is_split_layout(directory: str | pathlib.Path) -> bool :staticmethod: Return ``True`` if *directory* holds a pre-split train/test layout. .. py:method:: is_raw_layout(directory: str | pathlib.Path) -> bool :staticmethod: Return ``True`` if *directory* holds a single generated blob (not split). .. py:method:: load() -> tuple Return raw ``(data, labels, metadata)``, generating + caching on a miss. With no ``cache_dir`` the dataset is generated fresh. Otherwise the cache directory is checked first (first-look); on a hit the persisted blob is loaded without regeneration, on a miss it is generated and persisted. .. py:class:: FreqShapesDataset(name: str = 'freq_shapes', cache_dir: str | pathlib.Path | None = None, seed: int = 0, n_samples: int = 600, length: int = 500, class_freqs: tuple[int, Ellipsis] = (7, 19, 37), packet_len: int = 150, n_global_distractors: int = 3, n_local_distractors: int = 3, max_freq: int = 49, max_samples: int | None = None, sample_strategy: str = 'random', max_series_length: int | None = None, series_position: str = 'first') Bases: :py:obj:`SyntheticDataset` Localized wave-packet dataset for XAI time-frequency localization evaluation. Each sample is a sum of windowed sinusoids ("wave packets"). A multi-hot label marks which discriminative attributes are present; attribute ``k`` adds a packet at its characteristic frequency ``class_freqs[k]`` placed at a random time window. Non-discriminative distractor packets — global background sinusoids and shorter local packets, all at *other* frequencies — are added so the discriminative signal must be localized in both time and frequency. The per-sample metadata records the ground-truth regions **per class** (``ground_truth = {attr_index: [component, ...]}``) plus distractors (``non_discriminative``); a component is ``{"channel", "pos", "len", "freq", "phase"}``. This makes the dataset a ground truth for time-frequency attribution metrics (e.g. TimeFrequencyAUC). .. note:: The programmatic generator is **currently disabled** — :meth:`generate_dataset` is a placeholder that raises. ``freq_shapes`` ships a fixed, paper-faithful pre-split dataset committed under the synthetic cache dir (``cache/datasets/synthetic/freq_shapes/``); the runner loads it directly as a pre-split layout via ``load_saved_splits`` when ``cache_path`` is set. The constructor still accepts the original generation ``init_params`` so existing configs do not break, but they are inert. This is a temporary arrangement until the generator is validated. .. py:attribute:: n_samples :value: 600 .. py:attribute:: length :value: 500 .. py:attribute:: class_freqs :value: (7, 19, 37) .. py:attribute:: packet_len :value: 150 .. py:attribute:: n_global_distractors :value: 3 .. py:attribute:: n_local_distractors :value: 3 .. py:attribute:: max_freq :value: 49 .. py:method:: generate_dataset() -> tuple :abstractmethod: Disabled placeholder — the programmatic generator is turned off for now. ``freq_shapes`` ships a fixed pre-split dataset committed under the synthetic cache dir; the runner loads it directly (see the class docstring). This stub exists only to satisfy the :class:`SyntheticDataset` ABC. :raises NotImplementedError: Always — set ``cache_path`` so the runner loads the shipped pre-split layout instead of generating. .. py:data:: SYNTHETIC_DATASETS :type: dict[str, type] .. py:function:: register_synthetic_dataset(name: str, dataset_class: type) -> None Register a custom :class:`SyntheticDataset` subclass under *name*. The factory :func:`xai4tsc.data.load_dataset` dispatches a YAML ``dataset: `` entry to the registered class (checked before the path/UCR branches), so registering a name makes it usable from config. :param name: Registry key (matched against the YAML ``dataset`` field). :type name: str :param dataset_class: A :class:`SyntheticDataset` subclass. :type dataset_class: type .. py:function:: load_dataset(name: str, path: str | None = None, cache_dir: str | None = None, pad_series: bool = False, max_samples: int | None = None, sample_strategy: str = 'random', max_series_length: int | None = None, series_position: str = 'first', **kwargs: object) -> xai4tsc.data.base.DatasetBase Resolve to the correct :class:`DatasetBase` subclass. :param name: Human-readable dataset name (always required). If *name* is in the :data:`SYNTHETIC_DATASETS` registry, the matching :class:`SyntheticDataset` is returned (checked first) and generation params are forwarded as ``**kwargs``. :type name: str :param path: If provided, a :class:`LocalDataset` is returned pointing at *path*. If ``None`` (and *name* is not a synthetic dataset), a :class:`UcrUeaDataset` is returned. :type path: str | Path | None :param cache_dir: Passed through to :class:`UcrUeaDataset` when *path* is ``None``, stores UCR / UEA datasets to mitigate multiple downloads. :type cache_dir: Path | None :param pad_series: If ``True``, variable-length series are zero-padded to the maximum series length before returning data. Passed to the selected subclass. :type pad_series: bool :param max_samples: Subsample to at most this many samples before splitting. Prevents OOM on very large datasets (e.g. InsectWingbeat). :type max_samples: int | None :param sample_strategy: How to select the subset when *max_samples* is active: ``"random"`` (default), ``"first"``, ``"last"``, or ``"stratified"``. :type sample_strategy: str :param max_series_length: Truncate each series to at most this many timesteps. Applied before padding for variable-length datasets. :type max_series_length: int | None :param series_position: Which end of a long series to retain: ``"first"`` (default) or ``"last"``. :type series_position: str :param \*\*kwargs: Forwarded to the selected subclass constructor (e.g. ``download``, ``data_format``).