Stimuli delivery — BCI-Framework documentation

# Stimuli delivery

This interface use Brython and the Radian framework as backend for do the web development in a Python style, this interface inherits all features from Radian and extends the utilities with an specific ones.

## Bare minimum

    [ ]:

    from bci_framework.extensions.stimuli_delivery import StimuliAPI

    # Brython modules
    from browser import document, html
    from browser.widgets.dialog import InfoDialog

    # Main class must inherit `StimuliAPI`
    class StimuliDelivery(StimuliAPI):
        def __init__(self, *args, **kwargs):
            # Initialize `StimuliAPI` class
            super().__init__(*args, **kwargs)  # very importante line ;)
            document.clear()

            # -------------------------------------------------------------
            # main brython code
            document.select_one('body') <= html.H3('Hello world')

            button = html.BUTTON('click me')
            button.bind('click', lambda evt: InfoDialog('Hello', 'world'))
            document.select_one('body') <= button
            # -------------------------------------------------------------

    if __name__ == '__main__':
        # Create and run the server
        StimuliDelivery()

## Stimuli area and Dashboard

One of the main features is the possibility to make configurable experiments, in favor of this philosophy, by default they are builded both areas `self.stimuli_area` and `self.dashboard`.

    [ ]:

    # -------------------------------------------------------------
    # main brython code

    # Create a division for the stimuli_area and the dashboard
    self.stimuli_area <= html.H3('Stimuli area')
    self.dashboard <= html.H3('Dashboard')

    # Insert a cross in the middle of the stimuli area
    self.show_cross()

    # This area is used for external event processord
    self.show_synchronizer()
    # -------------------------------------------------------------

The `self.stimuli_area` at left attemp to be used to display stimuli and the `self.dashboard` on the right is for widgets and configurations.

## Widgets

All widgets and styles they are part of material-components-web with a custom framework implementation designed to display widgets and get values.

All widgets are available trought the `Widgets` submodule located in the module `bci_framework.extensions.stimuli_delivery.utils`.

    from bci_framework.extensions.stimuli_delivery.utils import Widgets as w

The following styles are used for all examples

    [ ]:

    flex = {'margin-bottom': '15px', 'display': 'flex', }
    flex_title = {'margin-top': '50px', 'margin-bottom': '10px', 'display': 'flex', }

### Typography

    [ ]:

    # -------------------------------------------------------------
    # main brython code

    self.dashboard <= w.label('headline1', typo='headline1', style=flex)
    self.dashboard <= w.label('headline2', typo='headline2', style=flex)
    self.dashboard <= w.label('headline3', typo='headline3', style=flex)
    self.dashboard <= w.label('headline4', typo='headline4', style=flex)
    self.dashboard <= w.label('headline5', typo='headline5', style=flex)
    self.dashboard <= w.label('headline6', typo='headline6', style=flex)
    self.dashboard <= w.label('body1', typo='body1', style=flex)
    self.dashboard <= w.label('body2', typo='body2', style=flex)
    self.dashboard <= w.label('subtitle1', typo='subtitle1', style=flex)
    self.dashboard <= w.label('subtitle2', typo='subtitle2', style=flex)
    self.dashboard <= w.label('caption', typo='caption', style=flex)
    self.dashboard <= w.label('button', typo='button', style=flex)
    self.dashboard <= w.label('overline', typo='overline', style=flex)
    # -------------------------------------------------------------

### Buttons

    [ ]:

        # -------------------------------------------------------------
        # main brython code
        self.dashboard <= w.label('Buttons', typo='headline4', style=flex_title)
        self.dashboard <= w.button('Button 1', style=flex, on_click=lambda: setattr(document.select_one('#for_button'), 'html', 'Button 1 pressed!'))
        self.dashboard <= w.button('Button 2', style=flex, on_click=self.on_button2)
        self.dashboard <= w.label(f'', id='for_button', typo=f'body1', style=flex)
        # -------------------------------------------------------------

    def on_button2(self):
        document.select_one('#for_button').html = 'Button 2 pressed!'

### Switch

    [ ]:

        # -------------------------------------------------------------
        # main brython code
        self.dashboard <= w.label('Switch', typo='headline4', style=flex_title)
        self.dashboard <= w.switch('Switch 1', checked=True, on_change=self.on_switch, id='my_swicth')
        self.dashboard <= w.label(f'', id='for_switch', typo=f'body1', style=flex)
        # -------------------------------------------------------------

    def on_switch(self, value):
        # value = self.widgets.get_value('my_swicth')
        document.select_one('#for_switch').html = f'Switch Changed: {value}'

### Checkbox

    [ ]:

        # -------------------------------------------------------------
        # main brython code
        self.dashboard <= w.label('Checkbox', typo='headline4', style=flex_title)
        self.dashboard <= w.checkbox('Checkbox', options=[[f'chb-{i}', False] for i in range(4)], on_change=self.on_checkbox, id='my_checkbox')
        self.dashboard <= w.label(f'', id='for_checkbox', typo=f'body1', style=flex)
        # -------------------------------------------------------------

    def on_checkbox(self):
        value = w.get_value('my_checkbox')
        document.select_one('#for_checkbox').html = f'Checkbox Changed: {value}'

### Radios

    [ ]:

        # -------------------------------------------------------------
        # main brython code
        self.dashboard <= w.label('Radios', typo='headline4', style=flex_title)
        self.dashboard <= w.radios('Radios', options=[[f'chb-{i}', f'chb-{i}'] for i in range(4)], on_change=self.on_radios, id='my_radios')
        self.dashboard <= w.label(f'', id='for_radios', typo=f'body1', style=flex)
        # -------------------------------------------------------------

    def on_radios(self):
        value = w.get_value('my_radios')
        document.select_one('#for_radios').html = f'Radios Changed: {value}'

### Select

    [ ]:

        # -------------------------------------------------------------
        # main brython code
        self.dashboard <= w.label('Select', typo='headline4', style=flex)
        self.dashboard <= w.select('Select', [[f'sel-{i}', f'sel-{i}'] for i in range(4)], on_change=self.on_select, id='my_select')
        self.dashboard <= w.label(f'', id='for_select', typo=f'body1', style=flex)
        # -------------------------------------------------------------

    def on_select(self, value):
        # value = w.get_value('my_select')
        document.select_one('#for_select').html = f'Select Changed: {value}'

### Sliders

    [ ]:

        # -------------------------------------------------------------
        # main brython code
        # Slider
        self.dashboard <= w.label('Slider', typo='headline4', style=flex)
        self.dashboard <= w.slider('Slider', min=1, max=10, step=0.1, value=5, on_change=self.on_slider, id='my_slider')
        self.dashboard <= w.label(f'', id='for_slider', typo=f'body1', style=flex)

        # Slider range
        self.dashboard <= w.label('Slider range', typo='headline4', style=flex)
        self.dashboard <= w.range_slider('Slider range', min=0, max=20, value_lower=5, value_upper=15, step=1, on_change=self.on_slider_range, id='my_range')
        self.dashboard <= w.label(f'', id='for_range', typo=f'body1', style=flex)
        # -------------------------------------------------------------

    def on_slider(self, value):
        # value = w.get_value('my_slider')
        document.select_one('#for_slider').html = f'Slider Changed: {value}'

    def on_slider_range(self, value):
        # value = w.get_value('my_slider')
        document.select_one('#for_range').html = f'Range Changed: {value}'

## Sound

### Tones

The `Tone` library allows playing single notes using the javascript `AudioContext` backend, the `duration` and the `gain` can also be configured.

    [ ]:

    from bci_framework.extensions.stimuli_delivery.utils import Tone as t

        # -------------------------------------------------------------
        # main brython code
        duration = 100
        gain = 0.5

        self.dashboard <= w.button('f#4', on_click=lambda: t('f#4', duration, gain), style={'margin': '0 15px'})
        self.dashboard <= w.button('D#0', on_click=lambda: t('D#0', duration, gain), style={'margin': '0 15px'})
        self.dashboard <= w.button('B2', on_click=lambda: t('B2', duration, gain), style={'margin': '0 15px'})
        # -------------------------------------------------------------

### Audio files

    Not implemented yet

## Pipelines

Pipelines consist of the controlled execution of methods with asynchronous timeouts.

Let’s assume that we have 4 view methods, each method could be a step needed to build a trial.

    [ ]:

    def view1(self, s1, r1):
        print(f'On view1: {s1=}, {r1=}')

    def view2(self, s1, r1):
        print(f'On view2: {s1=}, {r1=}')

    def view3(self, s1, r1):
        print(f'On view3: {s1=}, {r1=}')

    def view4(self, s1, r1):
        print(f'On view4: {s1=}, {r1=}\n')

The first rule is that all methods will have the same arguments, these arguments consist of the needed information to build a single trial.

Now, we need the `trials`, for example, here we define 3 trials (notice the arguments):

    [ ]:

    trials = [
        {'s1': 'Hola',  # Trial 1
               'r1': 91,
         },

        {'s1': 'Mundo',  # Trial 2
         'r1': 85,
         },

        {'s1': 'Python',  # Trial 3
         'r1': 30,
         },
    ]

And the `pipeline` consists of a list of sequential methods with a respective timeout `(method, timeout)`, if the `timeout` is a number then this will indicate the milliseconds until the next method call. If the `timeout` is a list, then a random (with uniform distribution) number between that range will be generated on each trial.

    [ ]:

    pipeline_trial = [
        (self.view1, 500),
        (self.view2, [500, 1500]),
        (self.view3, w.get_value('slider')),
        (self.view4, w.get_value('range')),
    ]

Finally, our pipeline can be executed with the method `self.run_pipeline`:

    [ ]:

    self.run_pipeline(pipeline_trial, trials)

    [ ]:

    On view1: s1=Hola, r1=91
    On view2: s1=Hola, r1=91
    On view3: s1=Hola, r1=91
    On view4: s1=Hola, r1=91

    On view1: s1=Mundo, r1=85
    On view2: s1=Mundo, r1=85
    On view3: s1=Mundo, r1=85
    On view4: s1=Mundo, r1=85

    On view1: s1=Python, r1=30
    On view2: s1=Python, r1=30
    On view3: s1=Python, r1=30
    On view4: s1=Python, r1=30

## Recording EEG automatically

If there is a current EEG streaming, the stimuli delivery can be configured to automatically start and stop the EEG recording with the methods `self.start_record()` and `self.stop_record()` respectively.

## Send Markers

The markers are used to synchronize events. The `self.send_marker` method is available all the time to stream markers through the streaming platform.

    [ ]:

    self.send_marker("MARKER")

The method self.send_marker works on the delivery views, so, if you have not an active remote presentation the markers will never send.

## Hardware-based event synchonization

In case that there is needed maximum precision about markers synchronization is possible to attach an external input directly to the analog inputs of OpenBCI.
The method `self.show_synchronizer()` do the trick, and is possible to configure the duration of the event with the `blink` argument in `self.send_marker`:

    [ ]:

    self.send_marker('RIGHT', blink=100)
    self.send_marker('LEFT', blink=200)

## Fixation cross

The fixation cross merely serves to center the subject’s gaze at the center of the screen. We can control the presence of this mark with the methods:

    [ ]:

    self.show_cross()
    self.hide_cross()

## Send annotations

In the same way that `markers`, the anotations (as defined in the EDF file format) can be streamed with the `self.send_annotation` method.

    [ ]:

    self.send_annotationn('Data record start')
    self.send_annotationn('The subject yawn', duration=5)

## Feedbacks

The `feedbacks`are used to comunicate the Data analysis and Data visualizations with the Stimuli Delivery platform. For this purpose, there is a predefined stream channel called `feedback`. The stimuli delivery is only able to read feedbacks, this is useful to develop neurofeedback applications.

The asynchronous handler can be configured with the `self.listen_feedbacks` method:

    [ ]:

    self.listen_feedbacks(self.on_feedback)

The target method requires the argument `command`.

    [ ]:

    def on_feedback(self, value):
        ...