
.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/plot_sleep_staging.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_auto_examples_plot_sleep_staging.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_plot_sleep_staging.py:


Sleep staging on the Sleep Physionet dataset
============================================

This tutorial shows how to train and test a sleep staging neural network with
Braindecode. We follow the approach of [1]_ on the openly accessible Sleep
Physionet dataset [1]_ [2]_.

References
----------
.. [1] Chambon, S., Galtier, M., Arnal, P., Wainrib, G. and Gramfort, A.
      (2018)A Deep Learning Architecture for Temporal Sleep Stage
      Classification Using Multivariate and Multimodal Time Series.
      IEEE Trans. on Neural Systems and Rehabilitation Engineering 26:
      (758-769)

.. [2] B Kemp, AH Zwinderman, B Tuk, HAC Kamphuisen, JJL Oberyé. Analysis of
       a sleep-dependent neuronal feedback loop: the slow-wave
       microcontinuity of the EEG. IEEE-BME 47(9):1185-1194 (2000).

.. [3] Goldberger AL, Amaral LAN, Glass L, Hausdorff JM, Ivanov PCh,
       Mark RG, Mietus JE, Moody GB, Peng C-K, Stanley HE. (2000)
       PhysioBank, PhysioToolkit, and PhysioNet: Components of a New
       Research Resource for Complex Physiologic Signals.
       Circulation 101(23):e215-e220

.. GENERATED FROM PYTHON SOURCE LINES 27-32

.. code-block:: default

    # Authors: Hubert Banville <hubert.jbanville@gmail.com>
    #
    # License: BSD (3-clause)









.. GENERATED FROM PYTHON SOURCE LINES 33-36

Loading and preprocessing the dataset
-------------------------------------


.. GENERATED FROM PYTHON SOURCE LINES 38-41

Loading
~~~~~~~


.. GENERATED FROM PYTHON SOURCE LINES 43-57

First, we load the data using the
:class:`braindecode.datasets.sleep_physionet.SleepPhysionet` class. We load
two recordings from two different individuals: we will use the first one to
train our network and the second one to evaluate performance (as in the `MNE`_
sleep staging example).

.. _MNE: https://mne.tools/stable/auto_tutorials/sample-datasets/plot_sleep.html

.. note::
   To load your own datasets either via MNE or from
   preprocessed X/y numpy arrays, see the `MNE Dataset
   Tutorial <./plot_mne_dataset_example.html>`__ and the `Numpy Dataset
   Tutorial <./plot_custom_dataset_example.html>`__.


.. GENERATED FROM PYTHON SOURCE LINES 57-64

.. code-block:: default


    from braindecode.datasets.sleep_physionet import SleepPhysionet

    dataset = SleepPhysionet(
        subject_ids=[0, 1], recording_ids=[1], crop_wake_mins=30)






.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    Using default location ~/mne_data for PHYSIONET_SLEEP...
    Extracting EDF parameters from /home/robintibor/mne_data/physionet-sleep-data/SC4001E0-PSG.edf...
    EDF file detected
    Setting channel info structure...
    Creating raw.info structure...
    Extracting EDF parameters from /home/robintibor/mne_data/physionet-sleep-data/SC4011E0-PSG.edf...
    EDF file detected
    Setting channel info structure...
    Creating raw.info structure...




.. GENERATED FROM PYTHON SOURCE LINES 65-68

Preprocessing
~~~~~~~~~~~~~


.. GENERATED FROM PYTHON SOURCE LINES 71-75

Next, we preprocess the raw data. We apply convert the data to microvolts and
apply a lowpass filter. We omit the downsampling step of [1]_ as the Sleep
Physionet data is already sampled at a lower 100 Hz.


.. GENERATED FROM PYTHON SOURCE LINES 75-89

.. code-block:: default


    from braindecode.datautil.preprocess import preprocess, Preprocessor

    high_cut_hz = 30

    preprocessors = [
        Preprocessor(lambda x: x * 1e6),
        Preprocessor('filter', l_freq=None, h_freq=high_cut_hz)
    ]

    # Transform the data
    preprocess(dataset, preprocessors)






.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    Reading 0 ... 2508000  =      0.000 ... 25080.000 secs...
    Filtering raw data in 1 contiguous segment
    Setting up low-pass filter at 30 Hz

    FIR filter parameters
    ---------------------
    Designing a one-pass, zero-phase, non-causal lowpass filter:
    - Windowed time-domain design (firwin) method
    - Hamming window with 0.0194 passband ripple and 53 dB stopband attenuation
    - Upper passband edge: 30.00 Hz
    - Upper transition bandwidth: 7.50 Hz (-6 dB cutoff frequency: 33.75 Hz)
    - Filter length: 45 samples (0.450 sec)

    Reading 0 ... 3261000  =      0.000 ... 32610.000 secs...
    Filtering raw data in 1 contiguous segment
    Setting up low-pass filter at 30 Hz

    FIR filter parameters
    ---------------------
    Designing a one-pass, zero-phase, non-causal lowpass filter:
    - Windowed time-domain design (firwin) method
    - Hamming window with 0.0194 passband ripple and 53 dB stopband attenuation
    - Upper passband edge: 30.00 Hz
    - Upper transition bandwidth: 7.50 Hz (-6 dB cutoff frequency: 33.75 Hz)
    - Filter length: 45 samples (0.450 sec)





.. GENERATED FROM PYTHON SOURCE LINES 90-93

Extract windows
~~~~~~~~~~~~~~~


.. GENERATED FROM PYTHON SOURCE LINES 96-97

We extract 30-s windows to be used in the classification task.

.. GENERATED FROM PYTHON SOURCE LINES 97-120

.. code-block:: default


    from braindecode.datautil.windowers import create_windows_from_events


    mapping = {  # We merge stages 3 and 4 following AASM standards.
        'Sleep stage W': 0,
        'Sleep stage 1': 1,
        'Sleep stage 2': 2,
        'Sleep stage 3': 3,
        'Sleep stage 4': 3,
        'Sleep stage R': 4
    }

    window_size_s = 30
    sfreq = 100
    window_size_samples = window_size_s * sfreq

    windows_dataset = create_windows_from_events(
        dataset, trial_start_offset_samples=0, trial_stop_offset_samples=0,
        window_size_samples=window_size_samples,
        window_stride_samples=window_size_samples, preload=True, mapping=mapping)






.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    Used Annotations descriptions: ['Sleep stage 1', 'Sleep stage 2', 'Sleep stage 3', 'Sleep stage 4', 'Sleep stage R', 'Sleep stage W']
    Adding metadata with 4 columns
    Replacing existing metadata with 4 columns
    837 matching events found
    No baseline correction applied
    0 projection items activated
    Loading data for 837 events and 3000 original time points ...
    0 bad epochs dropped
    Used Annotations descriptions: ['Sleep stage 1', 'Sleep stage 2', 'Sleep stage 3', 'Sleep stage 4', 'Sleep stage R', 'Sleep stage W']
    Adding metadata with 4 columns
    Replacing existing metadata with 4 columns
    1088 matching events found
    No baseline correction applied
    0 projection items activated
    Loading data for 1088 events and 3000 original time points ...
    0 bad epochs dropped




.. GENERATED FROM PYTHON SOURCE LINES 121-124

Window preprocessing
~~~~~~~~~~~~~~~~~~~


.. GENERATED FROM PYTHON SOURCE LINES 127-130

We also preprocess the windows by applying channel-wise z-score normalization
in each window.


.. GENERATED FROM PYTHON SOURCE LINES 130-136

.. code-block:: default


    from braindecode.datautil.preprocess import zscore

    preprocess(windows_dataset, [Preprocessor(zscore)])









.. GENERATED FROM PYTHON SOURCE LINES 137-140

Split dataset into train and valid
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


.. GENERATED FROM PYTHON SOURCE LINES 142-146

We can easily split the dataset using additional info stored in the
`description` attribute of :class:`braindecode.datasets.BaseDataset`,
in this case using the ``subject`` column. Here, we split the examples per subject.


.. GENERATED FROM PYTHON SOURCE LINES 146-156

.. code-block:: default


    splitted = windows_dataset.split('subject')
    train_set = splitted['0']
    valid_set = splitted['1']

    # Print number of examples per class
    print(train_set.datasets[0].windows)
    print(valid_set.datasets[0].windows)






.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    <Epochs |  837 events (all good), 0 - 29.99 sec, baseline off, ~38.3 MB, data loaded, with metadata,
     'Sleep stage 1': 58
     'Sleep stage 2': 250
     'Sleep stage 3': 220
     'Sleep stage 4': 220
     'Sleep stage R': 125
     'Sleep stage W': 184>
    <Epochs |  1088 events (all good), 0 - 29.99 sec, baseline off, ~49.8 MB, data loaded, with metadata,
     'Sleep stage 1': 109
     'Sleep stage 2': 562
     'Sleep stage 3': 105
     'Sleep stage 4': 105
     'Sleep stage R': 170
     'Sleep stage W': 142>




.. GENERATED FROM PYTHON SOURCE LINES 157-160

Create model
------------


.. GENERATED FROM PYTHON SOURCE LINES 162-166

We can now create the deep learning model. In this tutorial, we use the sleep
staging architecture introduced in [1]_, which is a four-layer convolutional
neural network.


.. GENERATED FROM PYTHON SOURCE LINES 166-195

.. code-block:: default


    import torch
    from braindecode.util import set_random_seeds
    from braindecode.models import SleepStagerChambon2018

    cuda = torch.cuda.is_available()  # check if GPU is available
    device = 'cuda' if torch.cuda.is_available() else 'cpu'
    if cuda:
        torch.backends.cudnn.benchmark = True
    # Set random seed to be able to reproduce results
    set_random_seeds(seed=87, cuda=cuda)

    n_classes = 5
    # Extract number of channels and time steps from dataset
    n_channels = train_set[0][0].shape[0]
    input_size_samples = train_set[0][0].shape[1]

    model = SleepStagerChambon2018(
        n_channels,
        sfreq,
        n_classes=n_classes,
        input_size_s=input_size_samples / sfreq
    )

    # Send model to GPU
    if cuda:
        model.cuda()









.. GENERATED FROM PYTHON SOURCE LINES 196-199

Training
--------


.. GENERATED FROM PYTHON SOURCE LINES 202-208

We can now train our network. :class:`braindecode.EEGClassifier` is a
braindecode object that is responsible for managing the training of neural
networks. It inherits from :class:`skorch.NeuralNetClassifier`, so the
training logic is the same as in
`Skorch <https://skorch.readthedocs.io/en/stable/>`__.


.. GENERATED FROM PYTHON SOURCE LINES 211-217

**Note**: We use different hyperparameters from [1]_, as
these hyperparameters were optimized on a different dataset (MASS SS3) and
with a different number of recordings. Generally speaking, it is
recommended to perform hyperparameter optimization if reusing this code on
a different dataset or with more recordings.


.. GENERATED FROM PYTHON SOURCE LINES 217-250

.. code-block:: default


    from skorch.helper import predefined_split
    from skorch.callbacks import EpochScoring
    from braindecode import EEGClassifier

    lr = 5e-4
    batch_size = 16
    n_epochs = 5

    train_bal_acc = EpochScoring(
        scoring='balanced_accuracy', on_train=True, name='train_bal_acc',
        lower_is_better=False)
    valid_bal_acc = EpochScoring(
        scoring='balanced_accuracy', on_train=False, name='valid_bal_acc',
        lower_is_better=False)
    callbacks = [('train_bal_acc', train_bal_acc),
                 ('valid_bal_acc', valid_bal_acc)]

    clf = EEGClassifier(
        model,
        criterion=torch.nn.CrossEntropyLoss,
        optimizer=torch.optim.Adam,
        train_split=predefined_split(valid_set),  # using valid_set for validation
        optimizer__lr=lr,
        batch_size=batch_size,
        callbacks=callbacks,
        device=device
    )
    # Model training for a specified number of epochs. `y` is None as it is already
    # supplied in the dataset.
    clf.fit(train_set, y=None, epochs=n_epochs)






.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

      epoch    train_bal_acc    train_loss    valid_bal_acc    valid_loss     dur
    -------  ---------------  ------------  ---------------  ------------  ------
          1           [36m0.2144[0m        [32m1.5027[0m           [35m0.2329[0m        [31m1.4245[0m  0.5432
          2           [36m0.3162[0m        [32m1.2601[0m           [35m0.4910[0m        [31m1.2843[0m  0.3833
          3           [36m0.4988[0m        [32m0.9767[0m           [35m0.5311[0m        [31m1.0593[0m  0.3653
          4           [36m0.5638[0m        [32m0.7986[0m           0.4689        [31m1.0150[0m  0.3696
          5           0.5532        [32m0.7489[0m           [35m0.5345[0m        [31m0.9308[0m  0.3697

    <class 'braindecode.classifier.EEGClassifier'>[initialized](
      module_=SleepStagerChambon2018(
        (spatial_conv): Conv2d(1, 2, kernel_size=(2, 1), stride=(1, 1))
        (feature_extractor): Sequential(
          (0): Conv2d(1, 8, kernel_size=(1, 50), stride=(1, 1), padding=(0, 25))
          (1): Identity()
          (2): ReLU()
          (3): MaxPool2d(kernel_size=(1, 13), stride=(1, 13), padding=0, dilation=1, ceil_mode=False)
          (4): Conv2d(8, 8, kernel_size=(1, 50), stride=(1, 1), padding=(0, 25))
          (5): Identity()
          (6): ReLU()
          (7): MaxPool2d(kernel_size=(1, 13), stride=(1, 13), padding=0, dilation=1, ceil_mode=False)
        )
        (fc): Sequential(
          (0): Dropout(p=0.25, inplace=False)
          (1): Linear(in_features=272, out_features=5, bias=True)
        )
      ),
    )



.. GENERATED FROM PYTHON SOURCE LINES 251-254

Plot results
------------


.. GENERATED FROM PYTHON SOURCE LINES 257-262

We use the history stored by Skorch during training to plot the performance of
the model throughout training. Specifically, we plot the loss and the balanced
misclassification rate (1 - balanced accuracy) for the training and validation
sets.


.. GENERATED FROM PYTHON SOURCE LINES 262-302

.. code-block:: default


    import matplotlib.pyplot as plt
    from matplotlib.lines import Line2D
    import pandas as pd

    # Extract loss and balanced accuracy values for plotting from history object
    df = pd.DataFrame(clf.history.to_list())
    df[['train_mis_clf', 'valid_mis_clf']] = 100 - df[
        ['train_bal_acc', 'valid_bal_acc']] * 100

    # get percent of misclass for better visual comparison to loss
    plt.style.use('seaborn-talk')
    fig, ax1 = plt.subplots(figsize=(8, 3))
    df.loc[:, ['train_loss', 'valid_loss']].plot(
        ax=ax1, style=['-', ':'], marker='o', color='tab:blue', legend=False,
        fontsize=14)

    ax1.tick_params(axis='y', labelcolor='tab:blue', labelsize=14)
    ax1.set_ylabel("Loss", color='tab:blue', fontsize=14)

    ax2 = ax1.twinx()  # instantiate a second axes that shares the same x-axis

    df.loc[:, ['train_mis_clf', 'valid_mis_clf']].plot(
        ax=ax2, style=['-', ':'], marker='o', color='tab:red', legend=False)
    ax2.tick_params(axis='y', labelcolor='tab:red', labelsize=14)
    ax2.set_ylabel('Balanced misclassification rate [%]', color='tab:red',
                   fontsize=14)
    ax2.set_ylim(ax2.get_ylim()[0], 85)  # make some room for legend
    ax1.set_xlabel('Epoch', fontsize=14)

    # where some data has already been plotted to ax
    handles = []
    handles.append(
        Line2D([0], [0], color='black', linewidth=1, linestyle='-', label='Train'))
    handles.append(
        Line2D([0], [0], color='black', linewidth=1, linestyle=':', label='Valid'))
    plt.legend(handles, [h.get_label() for h in handles], fontsize=14)
    plt.tight_layout()





.. image:: /auto_examples/images/sphx_glr_plot_sleep_staging_001.png
    :alt: plot sleep staging
    :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 303-305

Finally, we also display the confusion matrix and classification report:


.. GENERATED FROM PYTHON SOURCE LINES 305-317

.. code-block:: default


    from sklearn.metrics import confusion_matrix
    from sklearn.metrics import classification_report

    y_true = valid_set.datasets[0].windows.metadata['target'].values
    y_pred = clf.predict(valid_set)

    print(confusion_matrix(y_true, y_pred))

    print(classification_report(y_true, y_pred))






.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    [[120   0  15   2   5]
     [ 64   0  32   0  13]
     [ 29   0 487   6  40]
     [  0   0  22  83   0]
     [ 34   0 107   0  29]]
    /home/robintibor/anaconda3/envs/braindecode-test-apr-12/lib/python3.9/site-packages/sklearn/metrics/_classification.py:1248: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.
      _warn_prf(average, modifier, msg_start, len(result))
    /home/robintibor/anaconda3/envs/braindecode-test-apr-12/lib/python3.9/site-packages/sklearn/metrics/_classification.py:1248: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.
      _warn_prf(average, modifier, msg_start, len(result))
    /home/robintibor/anaconda3/envs/braindecode-test-apr-12/lib/python3.9/site-packages/sklearn/metrics/_classification.py:1248: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.
      _warn_prf(average, modifier, msg_start, len(result))
                  precision    recall  f1-score   support

               0       0.49      0.85      0.62       142
               1       0.00      0.00      0.00       109
               2       0.73      0.87      0.80       562
               3       0.91      0.79      0.85       105
               4       0.33      0.17      0.23       170

        accuracy                           0.66      1088
       macro avg       0.49      0.53      0.50      1088
    weighted avg       0.58      0.66      0.61      1088





.. GENERATED FROM PYTHON SOURCE LINES 318-326

Our model was able to perform reasonably well given the low amount of data
available, reaching a balanced accuracy of around 55% in a 5-class
classification task (chance-level = 20%) on held-out data.

To further improve performance, more recordings can be included in the
training set, and various modifications can be made to the model (e.g.,
aggregating the representation of multiple consecutive windows [1]_).



.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  5.045 seconds)


.. _sphx_glr_download_auto_examples_plot_sleep_staging.py:


.. only :: html

 .. container:: sphx-glr-footer
    :class: sphx-glr-footer-example



  .. container:: sphx-glr-download sphx-glr-download-python

     :download:`Download Python source code: plot_sleep_staging.py <plot_sleep_staging.py>`



  .. container:: sphx-glr-download sphx-glr-download-jupyter

     :download:`Download Jupyter notebook: plot_sleep_staging.ipynb <plot_sleep_staging.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
