yasa.EpochByEpochAgreement.summary#

EpochByEpochAgreement.summary(by_stage=False, ci_method=None, confidence=0.95, bootstrap_kwargs=None, **kwargs)[source]#

Return group-level agreement scores.

Parameters:
by_stagebool

If False (default), summary will include agreement scores derived from average-based metrics. If True, returned summary DataFrame will include agreement scores for each sleep stage, derived from one-vs-rest metrics.

ci_methodstr or None

Method used to compute a confidence interval of the mean of each metric across sessions (or subjects).

  • None (default) — no confidence interval is computed.

  • 'boot' — non-parametric bootstrap across sessions (or subjects). The same resamples are used for every metric (and stage), so each replicate remains internally consistent. Sessions in which a metric is undefined (see the zero_division parameter of get_agreement_bystage) are ignored, and replicates in which a metric has no defined value are excluded from its percentiles.

Added in version 0.8.0.

confidencefloat

Confidence level (between 0 and 1) of the confidence interval. Default is 0.95.

Added in version 0.8.0.

bootstrap_kwargsdict or None

Optional settings of the bootstrap procedure when ci_method='boot'. Valid keys are:

  • 'n_resamples' — number of bootstrap resamples (int, default 10000).

  • 'method' — 'BCa' (default) for bias-corrected and accelerated percentiles (Efron, 1987), 'percentile' for plain percentiles, or 'basic' for the reverse-percentile method. Metrics that are constant across all resamples fall back to plain percentiles. BCa can be unstable with few sessions (or subjects), and a warning is emitted below 20.

  • 'rng' — an integer seed or numpy.random.Generator for reproducible intervals (default None).

Added in version 0.8.0.

**kwargskey, value pairs

Additional keyword arguments are passed to pandas.DataFrame.groupby.agg. This can be used to customize the descriptive statistics returned.

Returns:
summarypandas.DataFrame

A pandas.DataFrame summarizing agreement scores across the entire dataset with descriptive statistics. Each row is an agreement metric (or a (stage, metric) pair if by_stage=True) and each column is a descriptive statistic (e.g., mean, standard deviation).

When ci_method is not None, two extra columns ci_lower and ci_upper are appended. They are left missing for support, which is not an agreement score.

With several nights per subject (subjects), the descriptive statistics and the bootstrap are computed across subjects rather than sessions, and support is the average number of epochs per night of each subject.

Changed in version 0.8.1: Support for several nights per subject (subjects).

Examples

The scores of the last get_agreement (or get_agreement_bystage if by_stage=True) call are summarized; if none was made, default scores are computed:

>>> import yasa
>>> ref_hyps = [yasa.simulate_hypnogram(tib=600, scorer="Human", seed=i) for i in range(5)]
>>> obs_hyps = [h.simulate_similar(scorer="YASA", seed=i) for i, h in enumerate(ref_hyps)]
>>> ebe = yasa.EpochByEpochAgreement(ref_hyps, obs_hyps)
>>> _ = ebe.get_agreement()
>>> ebe.summary().round(2)
               mad   mean   std    min  median    max
metric
accuracy      5.30  28.58  6.27  21.42   30.58  35.08
balanced_acc  4.13  24.16  5.85  16.85   23.87  32.68
kappa         0.05   0.04  0.07  -0.06    0.06   0.14
mcc           0.05   0.05  0.07  -0.06    0.06   0.14
precision     6.24  28.26  7.33  20.29   30.64  34.93
f1            5.89  27.97  6.88  20.50   30.56  34.55

To control the descriptive statistics included as columns:

>>> ebe.summary(func=["count", "mean", "sem"]).round(2)
              count   mean   sem
metric
accuracy        5.0  28.58  2.80
balanced_acc    5.0  24.16  2.62
kappa           5.0   0.04  0.03
mcc             5.0   0.05  0.03
precision       5.0  28.26  3.28
f1              5.0  27.97  3.07

To add a bootstrap confidence interval of the mean across sessions, pass ci_method.

>>> ebe.summary(
...     ci_method="boot",
...     bootstrap_kwargs={"method": "percentile", "n_resamples": 1000, "rng": 0},
...     func=["mean"],
... ).round(2)
               mean  ci_lower  ci_upper
metric
accuracy      28.58     23.68     33.28
balanced_acc  24.16     20.08     29.09
kappa          0.04     -0.01      0.10
mcc            0.05     -0.01      0.10
precision     28.26     22.50     34.03
f1            27.97     22.60     33.13