Framework

The experiment runner is the standalone use case for xai4tsc. Clone the repository, write a YAML config, and run a full evaluation pipeline from the command line — no Python scripting required. Results are saved automatically to CSV files grouped by dataset and model.

Installation

git clone https://github.com/TimeXAI-group/XAI4TSC.git
cd XAI4TSC
conda env create          # picks up environment.yml
conda activate xai4tsc
python -m venv .venv
source .venv/bin/activate
poetry install

Getting Started

Copy experiment_runner/configs/example.yaml and edit it for your datasets, models, explainers, and metrics.

Run the experiment as a module from the repository root:

python -m experiment_runner.main --conf path/to/your/config.yaml
python -m experiment_runner.main --conf path/to/your/config.yaml --debug

Results are written under results_rel_path/<experiment_name>/ (the experiment_name from your config), organised as:

results_rel_path/
└── <experiment_name>/
    ├── metrics.csv            # all results
    ├── experiment.log         # run log
    └── <dataset>/
        ├── metrics.csv        # per-dataset results
        └── <model>/
            ├── metrics.csv    # per-model results
            ├── <model>_epoch_<n>.pt   # best checkpoint
            └── explanations/  # per-sample relevance plots

See experiment_runner/configs/master.yaml for a fully annotated reference of every available config key and its default value.

Example

The shipped experiment_runner/configs/example.yaml is the go-to demo and the config the runner uses by default (python -m experiment_runner.main). It needs no download: the synthetic freq_shapes dataset is shared with the repository. It trains two model architectures (LeNet and FCN), explains them with three time-domain methods (Integrated Gradients, Guided Backpropagation, and TSHAP), and scores those explanations with the Complexity and Pixel-Flipping metrics:

# =============================================================================
# example.yaml — the go-to demo for xai4tsc (time domain).
#
# This is the config the runner uses by default:
#     python -m experiment_runner.main
#
# It needs no download: the synthetic `freq_shapes` dataset is generated locally
# (and cached). It trains TWO model architectures, explains them with THREE
# time-domain XAI methods, and scores those explanations with time-domain
# metrics.
#
# For the frequency / time-frequency counterpart, see `example_frequency.yaml`.
# For an annotated reference of every available option, see `master.yaml`.
# =============================================================================
experiment_name: "example_time"

# Generated datasets, splits, and checkpoints are cached here and reused.
cache_path: "./cache/"

general:
  reproducible: true
  seed: 2025
  device: "use_available"

# --- Data: synthetic localized wave-packet dataset (no download) ---
# `freq_shapes` ships a fixed, paper-faithful pre-split dataset committed under the
# synthetic cache dir (cache/datasets/synthetic/freq_shapes/). With `cache_path` set
# the runner loads that pre-split layout directly, preserving the original
# train/val/test split — so `train_split`/`val_split` and any generation
# `init_params` below are inert for this dataset (the generator is disabled for now).
data_config:
  default_settings:
    train_split: 0.8
    val_split: 0.1
  datasets:
    - dataset: "freq_shapes"        # SyntheticDataset registry key
      encode: "multihot"            # collapse multi-hot combos to class indices

# --- Models: two different architectures, both trained ---
train_config:
  default_settings:
    hyperparams:
      epochs: 10
      batchsize: 32
      loss_func: "CrossEntropy"
      optimizer: "adam"
      learn_rate: 0.001
      patience: 3
      save_best: true
    train: "till_epoch"
  models:
    - model: "LeNet"                # smallest built-in CNN
    - model: "FCN"                  # fully-convolutional baseline
    # in_channels / num_classes are auto-detected per dataset.

# --- Explainers: three time-domain methods ---
explanation_config:
  default_settings:
    target: "predicted"
    samples:
      type: "random"
      count: 5
  explainers:
    - method: Integrated_Gradients
    - method: Guided_Backpropagation
    - method: TSHAP                 # exact 2-player Shapley over time windows
      window_length: 0.1           # fraction of T per window
      stride: 10                   # compute every 10th window start (bounds cost)
      perturb_baseline: "centroid"
      background_data: train_set

# --- Metrics: time-domain ---
evaluation_config:
  default_settings:
    metric_class_params:
      normalise: true
      abs: true
      disable_warnings: true
      return_aggregate: true
    metric_call_params:
      softmax: false
  metrics:
    - metric: "Complexity"          # cheap: counts non-zero attributions
    - metric: "Pixel-Flipping"      # faithfulness via progressive perturbation

results_rel_path: "./experiments/results/"

For the frequency / time-frequency counterpart — FreqRISE and the frequency metrics on the same dataset — see experiment_runner/configs/example_frequency.yaml. Full archive sweeps are in ucr_benchmark.yaml and uea_benchmark.yaml.

API Reference

The experiment runner is built around a small set of modules:

experiment_runner.main — entry point, orchestrates the pipeline
experiment_runner.config — config resolution and validation
experiment_runner.cache — caching layer for results
experiment_runner.evaluate — evaluation routines
experiment_runner.explain — explanation generation
experiment_runner.log_setup — logging configuration

See the full experiment_runner for complete details.