yasa.EpochByEpochAgreement.get_confusion_matrix_proportional#

EpochByEpochAgreement.get_confusion_matrix_proportional(ci_method='boot', confidence=0.95, bootstrap_kwargs=None, formatted=False, decimals=1)[source]#

Return the group-level proportional confusion (error) matrix, i.e. the mean, standard deviation, and confidence interval across sessions of the row-normalized per-session confusion matrices, as reported in Menghini et al. (2021).

For each session, the confusion matrix is first normalized by row, so that each cell is the percentage of reference-scorer epochs of a given stage that the observed scorer assigned to each stage (each row sums to 100). These per-session percentages are then averaged across sessions (subject-then-group averaging), so that every session contributes equally regardless of its duration. This differs from get_confusion_matrix(agg_func="sum"), which pools all epochs together.

Rows of the reference stage that are absent from a session (e.g. no N3 sleep in a given night) are undefined (0 / 0) and are excluded from the mean, SD, and CI of that row rather than being counted as zeros. The number of contributing sessions is returned in the n_sessions column.

Added in version 0.8.0.

Parameters:
ci_methodstr or None

Method used to compute the confidence interval of the group mean of each cell.

  • 'boot' (default) — non-parametric bootstrap across sessions (i.e., sessions are resampled with replacement, a “participant bootstrap”). The same resampled sessions are used for all cells, so that each bootstrapped matrix remains row-normalized. Replicates in which a reference stage is absent from every resampled session are undefined and are excluded from the percentiles of that row.

  • 'param' — parametric interval mean ± t × SD / sqrt(n) based on a Student t-distribution with n - 1 degrees of freedom, clipped to [0, 100].

  • None — no confidence interval is computed.

confidencefloat

Confidence level (between 0 and 1) of the confidence interval. Default is 0.95.

bootstrap_kwargsdict or None

Optional settings of the bootstrap procedure when ci_method='boot'. Valid keys are:

  • 'n_resamples' — number of bootstrap resamples (int, default 1000).

  • 'method' — 'BCa' (default) for bias-corrected and accelerated percentiles (Efron, 1987), which corrects for the skewness and bias of the bootstrap distribution but can be unstable with few sessions (e.g. fewer than 20); 'percentile' for plain percentiles; or 'basic' for the reverse-percentile method used by the reference pipeline of Menghini et al. (2021). Cells that are constant across all resamples (e.g. a stage that is never confused) fall back to plain percentiles.

  • 'rng' — an integer seed or numpy.random.Generator for reproducible intervals (default None).

formattedbool

If False (default), return the numeric statistics in long format. If True, return a square, human-readable matrix of strings formatted as "mean (SD) [lower, upper]" (or "mean (SD)" when ci_method=None).

decimalsint

Number of decimal places used when formatted=True. Default is 1.

Returns:
conf_matrpandas.DataFrame

If formatted=False, a long-format DataFrame with a MultiIndex of (reference stage, observed stage) pairs as rows and columns mean, std, ci_lower, ci_upper (the latter two only when ci_method is not None), and n_sessions. All values except n_sessions are percentages (0-100). Use e.g. conf_matr["mean"].unstack() to get the mean matrix in square format.

If formatted=True, a square DataFrame with stages from the reference scorer as index and stages from the observed scorer as columns, where each cell is a formatted string.

Examples

>>> import yasa
>>> ref_hyps = [yasa.simulate_hypnogram(tib=600, scorer="Human", seed=i) for i in range(10)]
>>> obs_hyps = [h.simulate_similar(scorer="YASA", seed=i) for i, h in enumerate(ref_hyps)]
>>> ebe = yasa.EpochByEpochAgreement(ref_hyps, obs_hyps)
>>> ebe.get_confusion_matrix_proportional(ci_method="param").head(5).round(1)
            mean   std  ci_lower  ci_upper  n_sessions
Human YASA
WAKE  WAKE  31.6   9.3      25.0      38.3          10
      N1    12.2   4.8       8.8      15.7          10
      N2    35.1  11.0      27.2      43.0          10
      N3     9.9   8.0       4.2      15.7          10
      REM   11.1  10.0       3.9      18.2          10

>>> ebe.get_confusion_matrix_proportional(ci_method=None, formatted=True)
YASA          WAKE          N1           N2           N3          REM
Human
WAKE    31.6 (9.3)  12.2 (4.8)  35.1 (11.0)    9.9 (8.0)  11.1 (10.0)
N1      21.3 (9.5)  16.2 (8.1)   35.7 (9.5)  14.1 (11.6)   12.7 (9.8)
N2     22.0 (10.5)  11.7 (3.1)  38.5 (11.8)  15.1 (10.9)  12.7 (11.8)
N3     22.9 (13.1)   9.7 (5.4)  37.2 (12.4)  21.9 (18.2)   8.4 (11.8)
REM    20.9 (20.4)   9.3 (5.4)  40.0 (13.0)  14.7 (16.0)  15.0 (17.9)

With the default bootstrap CI (ci_method="boot"), pass rng for reproducibility:

>>> ebe.get_confusion_matrix_proportional(formatted=True, bootstrap_kwargs={"rng": 1}).loc[
...     "WAKE", "WAKE"
... ]
'31.6 (9.3) [26.6, 38.0]'