yasa.EpochByEpochAgreement.summary#

EpochByEpochAgreement.summary(by_stage=False, ci_method=None, confidence=0.95, bootstrap_kwargs=None, **kwargs)[source]#

Return group-level agreement scores.

Parameters:
by_stagebool

If False (default), summary will include agreement scores derived from average-based metrics. If True, returned summary DataFrame will include agreement scores for each sleep stage, derived from one-vs-rest metrics.

ci_methodstr or None

Method used to compute a confidence interval of the mean of each metric across sessions.

  • None (default) — no confidence interval is computed.

  • 'boot' — non-parametric bootstrap across sessions. The same resampled sessions are used for every metric (and stage), so each replicate remains internally consistent. Sessions in which a metric is undefined (see the zero_division parameter of get_agreement_bystage) are ignored, and replicates in which a metric has no defined value are excluded from its percentiles.

Added in version 0.8.0.

confidencefloat

Confidence level (between 0 and 1) of the confidence interval. Default is 0.95.

Added in version 0.8.0.

bootstrap_kwargsdict or None

Optional settings of the bootstrap procedure when ci_method='boot'. Valid keys are:

  • 'n_resamples' — number of bootstrap resamples (int, default 10000).

  • 'method' — 'BCa' (default) for bias-corrected and accelerated percentiles (Efron, 1987), 'percentile' for plain percentiles, or 'basic' for the reverse-percentile method. Metrics that are constant across all resamples fall back to plain percentiles. BCa can be unstable with few sessions, and a warning is emitted below 20 sessions.

  • 'rng' — an integer seed or numpy.random.Generator for reproducible intervals (default None).

Added in version 0.8.0.

**kwargskey, value pairs

Additional keyword arguments are passed to pandas.DataFrame.groupby.agg. This can be used to customize the descriptive statistics returned.

Returns:
summarypandas.DataFrame

A pandas.DataFrame summarizing agreement scores across the entire dataset with descriptive statistics. Each row is an agreement metric (or a (stage, metric) pair if by_stage=True) and each column is a descriptive statistic (e.g., mean, standard deviation).

When ci_method is not None, two extra columns ci_lower and ci_upper are appended. They are left missing for support, which is not an agreement score.

Examples

The scores of the last get_agreement (or get_agreement_bystage if by_stage=True) call are summarized; if none was made, default scores are computed:

>>> import yasa
>>> ref_hyps = [yasa.simulate_hypnogram(tib=600, scorer="Human", seed=i) for i in range(5)]
>>> obs_hyps = [h.simulate_similar(scorer="YASA", seed=i) for i, h in enumerate(ref_hyps)]
>>> ebe = yasa.EpochByEpochAgreement(ref_hyps, obs_hyps)
>>> _ = ebe.get_agreement()
>>> ebe.summary().round(2)
               mad   mean   std    min  median    max
metric
accuracy      5.30  28.58  6.27  21.42   30.58  35.08
balanced_acc  4.13  24.16  5.85  16.85   23.87  32.68
kappa         0.05   0.04  0.07  -0.06    0.06   0.14
mcc           0.05   0.05  0.07  -0.06    0.06   0.14
precision     6.24  28.26  7.33  20.29   30.64  34.93
f1            5.89  27.97  6.88  20.50   30.56  34.55

To control the descriptive statistics included as columns:

>>> ebe.summary(func=["count", "mean", "sem"]).round(2)
              count   mean   sem
metric
accuracy        5.0  28.58  2.80
balanced_acc    5.0  24.16  2.62
kappa           5.0   0.04  0.03
mcc             5.0   0.05  0.03
precision       5.0  28.26  3.28
f1              5.0  27.97  3.07

To add a bootstrap confidence interval of the mean across sessions, pass ci_method.

>>> ebe.summary(
...     ci_method="boot",
...     bootstrap_kwargs={"method": "percentile", "n_resamples": 1000, "rng": 0},
...     func=["mean"],
... ).round(2)
               mean  ci_lower  ci_upper
metric
accuracy      28.58     23.68     33.28
balanced_acc  24.16     20.08     29.09
kappa          0.04     -0.01      0.10
mcc            0.05     -0.01      0.10
precision     28.26     22.50     34.03
f1            27.97     22.60     33.13
On this page