:tocdepth: 0








.. _/plugins/execute:

Execute Plugins
~~~~~~~~~~~~~~~



.. include:: execute-header.inc.rst

.. _/plugins/execute/common-keys:

Common Keys
-----------



The following keys are accepted by all plugins of the ``execute`` step.


name
    The name of the step phase.

    Environment variable: ``TMT_PLUGIN_EXECUTE_TMT_NAME``

    In plan metadata:

    .. code-block:: yaml

       name:

    On command-line:

    .. code-block:: shell

       --name ...
       export TMT_PLUGIN_EXECUTE_TMT_NAME=...



order
    Order in which the phase should be handled.

    Default: ``50``

    Environment variable: ``TMT_PLUGIN_EXECUTE_TMT_ORDER``

    In plan metadata:

    .. code-block:: yaml

       order:

    On command-line:

    .. code-block:: shell

       --order ...
       export TMT_PLUGIN_EXECUTE_TMT_ORDER=...



summary
    Concise summary describing purpose of the phase.

    Default: *not set*

    Environment variable: ``TMT_PLUGIN_EXECUTE_TMT_SUMMARY``

    In plan metadata:

    .. code-block:: yaml

       summary:

    On command-line:

    .. code-block:: shell

       --summary ...
       export TMT_PLUGIN_EXECUTE_TMT_SUMMARY=...



when
    If specified, phase is run only if any rule matches plan context.

    Default: *not set*

    Environment variable: ``TMT_PLUGIN_EXECUTE_TMT_WHEN``

    In plan metadata:

    .. code-block:: yaml

       when: RULE


    On command-line:

    .. code-block:: shell

       --when RULE
       export TMT_PLUGIN_EXECUTE_TMT_WHEN=RULE



where
    Run this phase on given guest or guests with the given role only.

    Default: *not set*

    Environment variable: ``TMT_PLUGIN_EXECUTE_TMT_WHERE``

    In plan metadata:

    .. code-block:: yaml

       where: GUEST|ROLE


    On command-line:

    .. code-block:: shell

       --where 'GUEST|ROLE'
       export TMT_PLUGIN_EXECUTE_TMT_WHERE='GUEST|ROLE'



.. _/plugins/execute/tmt:

tmt
---



.. warning::

    Please, be aware that the documentation below is a work in progress. We are
    working on fixing it, adding missing bits and generally making it better.
    Also, it was originally used for command line help only, therefore the
    formatting is often suboptimal.

Use the internal tmt executor to execute tests.

The internal tmt executor runs tests on the guest one by one directly
from the tmt code which shows testing :tmt:story:`/stories/cli/steps/execute/progress`
and supports :tmt:story:`/stories/cli/steps/execute/interactive` debugging as well.
This is the default execute step implementation. Test result is based on the
script exit code (for shell tests) or the results file (for beakerlib tests).

The executor provides the following shell scripts which can be used by the tests
for certain operations.

``tmt-file-submit`` - archive the given file in the tmt test data directory.
See the :tmt:story:`/stories/features/report-log` section for more details.

``tmt-reboot`` - soft reboot the machine from inside the test. After reboot
the execution starts from the test which rebooted the machine.
Use ``tmt-reboot -s`` for systemd soft-reboot which preserves the kernel
and hardware state while restarting userspace only.
An environment variable ``TMT_REBOOT_COUNT`` is provided which
the test can use to handle the reboot. The variable holds the
number of reboots performed by the test. For more information
see the :tmt:story:`/stories/features/reboot` feature documentation.

``tmt-report-result`` - generate a result report file from inside the test.
Can be called multiple times by the test. The generated report
file will be overwritten if a higher hierarchical result is
reported by the test. The hierarchy is as follows:
SKIP, PASS, WARN, FAIL. For more information see the
:tmt:story:`/stories/features/report-result` feature documentation.

``tmt-abort`` - generate an abort file from inside the test. This will
set the current test result to failed and terminate
the execution of subsequent tests. For more information see the
:tmt:story:`/stories/features/abort` feature documentation.

The scripts are hosted by default in the ``/usr/local/bin`` directory, except
for guests using ``rpm-ostree``, where ``/var/lib/tmt/scripts`` is used.
The directory can be forced using the ``TMT_SCRIPTS_DIR`` environment variable.
Note that for guests using ``rpm-ostree``, the directory is added to
executable paths using the system-wide ``/etc/profile.d/tmt.sh`` profile script.

.. warning::

    Please be aware that for guests using ``rpm-ostree``
    the provided scripts will only be available in a shell that
    loads the profile scripts. This is the default for
    ``bash``-like shells, but might not work for others.




Configuration
^^^^^^^^^^^^^



See also :ref:`Common Keys</plugins/execute/common-keys>` accepted by the plugin.


duration
    The maximum time test specified by ``script`` is allowed to run.
    This option does not affect the duration defined in test metadata.

    Default: ``1h``

    Environment variable: ``TMT_PLUGIN_EXECUTE_TMT_DURATION``

    In plan metadata:

    .. code-block:: yaml

       duration:

    On command-line:

    .. code-block:: shell

       --duration ...
       export TMT_PLUGIN_EXECUTE_TMT_DURATION=...



exit-first
    Stop execution after the first test failure or error.

    Default: ``false``

    Environment variable: ``TMT_PLUGIN_EXECUTE_TMT_EXIT_FIRST``

    In plan metadata:

    .. code-block:: yaml

       exit-first: true|false

    On command-line:

    .. code-block:: shell

       --exit-first
       export TMT_PLUGIN_EXECUTE_TMT_EXIT_FIRST=1|0


ignore-duration
    Ignore test duration value and allow test to run forever.
    Can be set by environment variable even when step is not
    specified on the commandline. This environment variable
    will be replaced by fmf config file or CLI arguments.

    Default: ``false``

    Environment variable: ``TMT_PLUGIN_EXECUTE_TMT_IGNORE_DURATION``

    In plan metadata:

    .. code-block:: yaml

       ignore-duration: true|false

    On command-line:

    .. code-block:: shell

       --ignore-duration
       export TMT_PLUGIN_EXECUTE_TMT_IGNORE_DURATION=1|0


interactive
    Run tests in interactive mode, i.e. with input and output streams
    shared with tmt itself. This allows input to be passed to tests
    via stdin, e.g. responding to password prompts. Test output in this
    mode is not captured, and ``duration`` has no effect.

    Default: ``false``

    Environment variable: ``TMT_PLUGIN_EXECUTE_TMT_INTERACTIVE``

    In plan metadata:

    .. code-block:: yaml

       interactive: true|false

    On command-line:

    .. code-block:: shell

       --interactive
       export TMT_PLUGIN_EXECUTE_TMT_INTERACTIVE=1|0


no-progress-bar
    Disable interactive progress bar showing the current test.

    Default: ``false``

    Environment variable: ``TMT_PLUGIN_EXECUTE_TMT_NO_PROGRESS_BAR``

    In plan metadata:

    .. code-block:: yaml

       no-progress-bar: true|false

    On command-line:

    .. code-block:: shell

       --no-progress-bar
       export TMT_PLUGIN_EXECUTE_TMT_NO_PROGRESS_BAR=1|0


restraint-compatible
    Run tests in the restraint-compatible mode. Enable this if
    your tests depend on the restraint scripts such as
    ``rstrnt-report-result`` or ``rstrnt-report-log``.
    Currently this option only affects whether the
    ``$OUTPUTFILE`` variable is respected, but in the future it
    will be used to enable/disable all restraint compatibility
    features.

    Default: ``false``

    Environment variable: ``TMT_PLUGIN_EXECUTE_TMT_RESTRAINT_COMPATIBLE``

    In plan metadata:

    .. code-block:: yaml

       restraint-compatible: true|false

    On command-line:

    .. code-block:: shell

       --restraint-compatible
       export TMT_PLUGIN_EXECUTE_TMT_RESTRAINT_COMPATIBLE=1|0


script
    Execute arbitrary shell commands and check their exit
    code which is used as a test result. The ``script`` field
    is provided to cover simple test use cases only and must
    not be combined with the :tmt:story:`/spec/plans/discover` step
    which is more suitable for more complex test scenarios.

    Default shell options are applied to the script, see
    :tmt:story:`/spec/tests/test` for more details. The default
    :tmt:story:`/spec/tests/duration` for tests defined directly
    under the execute step is ``1h``. Use the ``duration``
    attribute to modify the default limit.

    Default: *not set*

    Environment variable: ``TMT_PLUGIN_EXECUTE_TMT_SCRIPT``

    In plan metadata:

    .. code-block:: yaml

       script: SCRIPT


    On command-line:

    .. code-block:: shell

       --script SCRIPT
       export TMT_PLUGIN_EXECUTE_TMT_SCRIPT=SCRIPT



----


.. _/plugins/execute/upgrade:

upgrade
-------



.. warning::

    Please, be aware that the documentation below is a work in progress. We are
    working on fixing it, adding missing bits and generally making it better.
    Also, it was originally used for command line help only, therefore the
    formatting is often suboptimal.

Perform system upgrade during testing.

In order to enable developing tests for upgrade testing, we need to provide
a way how to execute these tests easily. This does not cover unit tests for
individual actors but rather system tests which verify
the whole upgrade story.

The upgrade executor runs the discovered tests (using the internal
executor), then performs a set of upgrade tasks from a remote
repository, and finally, re-runs the tests on the upgraded guest.

The ``IN_PLACE_UPGRADE`` environment variable is set during the test
execution to differentiate between the stages of the test. It is set
to ``old`` during the first execution and ``new`` during the second
execution. Test names are prefixed with this value to make the names
unique. Based on this variable, the test can perform appropriate actions.

* ``old``: setup, test
* ``new``: test, cleanup
* ``without``: setup, test, cleanup

The upgrade tasks performing the actual system upgrade are taken
from a remote repository (specified by the ``url`` key) based on an upgrade
path (e.g. ``fedora35to36``) or other filters (e.g. specified by the
``filter`` key). If both ``upgrade-path`` and extra filters are specified,
the discover keys in the remote upgrade path plan are overridden by the
filters specified in the local plan.

The upgrade path must correspond to a plan name in the
remote repository whose discover step selects tests (upgrade tasks)
performing the upgrade. Currently, selection of upgrade tasks in the remote
repository can be done using both fmf and shell discover method.
If the ``url`` is not provided, upgrade path and upgrade tasks are taken from
the current repository. The supported keys in discover are:

* ``ref``
* ``filter``
* ``exclude``
* ``tests``
* ``test``

The environment variables defined in the remote upgrade path plan are
passed to the upgrade tasks when they are executed. An example of an
upgrade path plan (in the remote repository):

.. code-block:: yaml

    discover: # Selects appropriate upgrade tasks (L1 tests)
        how: fmf
        filter: "tag:fedora"
    environment: # This is passed to upgrade tasks
        SOURCE: 35
        TARGET: 36
    execute:
        how: tmt

If no upgrade path is specified in the plan, the tests (upgrade tasks)
are selected based on the configuration of the upgrade plugin
(e.g. based on the filter in its configuration).

If these two possible ways of specifying upgrade tasks are combined,
the remote discover plan is used but its options are overridden
with the values specified locally.

The same options and config keys and values can be used as in the
internal executor.

Minimal execute config example with an upgrade path:

.. code-block:: yaml

    execute:
        how: upgrade
        url: https://github.com/teemtee/upgrade
        upgrade-path: /paths/fedora35to36

Execute config example without an upgrade path:

.. code-block:: yaml

    execute:
        how: upgrade
        url: https://github.com/teemtee/upgrade
        filter: "tag:fedora"

.. code-block:: yaml

    # A simple beakerlib test using the $IN_PLACE_UPGRADE variable
    . /usr/share/beakerlib/beakerlib.sh || exit 1

    VENV_PATH=/var/tmp/venv_test

    rlJournalStart
        # Perform the setup only for the old distro
        if [[ "$IN_PLACE_UPGRADE" !=  "new" ]]; then
            rlPhaseStartSetup
                rlRun "python3.9 -m venv $VENV_PATH"
                rlRun "$VENV_PATH/bin/pip install pyjokes"
            rlPhaseEnd
        fi

        # Execute the test for both old & new distro
        rlPhaseStartTest
            rlAsssertExists "$VENV_PATH/bin/pyjoke"
            rlRun "$VENV_PATH/bin/pyjoke"
        rlPhaseEnd

        # Skip the cleanup phase when on the old distro
        if [[ "$IN_PLACE_UPGRADE" !=  "old" ]]; then
            rlPhaseStartCleanup
                rlRun "rm -rf $VENV_PATH"
            rlPhaseEnd
        fi
    rlJournalEnd




Configuration
^^^^^^^^^^^^^



See also :ref:`Common Keys</plugins/execute/common-keys>` accepted by the plugin.


duration
    The maximum time test specified by ``script`` is allowed to run.
    This option does not affect the duration defined in test metadata.

    Default: ``1h``

    Environment variable: ``TMT_PLUGIN_EXECUTE_UPGRADE_DURATION``

    In plan metadata:

    .. code-block:: yaml

       duration:

    On command-line:

    .. code-block:: shell

       --duration ...
       export TMT_PLUGIN_EXECUTE_UPGRADE_DURATION=...



exclude
    Exclude a regular expression from search result.

    Default: *not set*

    Environment variable: ``TMT_PLUGIN_EXECUTE_UPGRADE_EXCLUDE``

    In plan metadata:

    .. code-block:: yaml

       exclude: REGEXP


    On command-line:

    .. code-block:: shell

       --exclude REGEXP
       export TMT_PLUGIN_EXECUTE_UPGRADE_EXCLUDE=REGEXP



exit-first
    Stop execution after the first test failure or error.

    Default: ``false``

    Environment variable: ``TMT_PLUGIN_EXECUTE_UPGRADE_EXIT_FIRST``

    In plan metadata:

    .. code-block:: yaml

       exit-first: true|false

    On command-line:

    .. code-block:: shell

       --exit-first
       export TMT_PLUGIN_EXECUTE_UPGRADE_EXIT_FIRST=1|0


filter
    Include only tests matching the filter.

    Default: *not set*

    Environment variable: ``TMT_PLUGIN_EXECUTE_UPGRADE_FILTER``

    In plan metadata:

    .. code-block:: yaml

       filter: FILTERS


    On command-line:

    .. code-block:: shell

       --filter FILTERS
       export TMT_PLUGIN_EXECUTE_UPGRADE_FILTER=FILTERS



ignore-duration
    Ignore test duration value and allow test to run forever.
    Can be set by environment variable even when step is not
    specified on the commandline. This environment variable
    will be replaced by fmf config file or CLI arguments.

    Default: ``false``

    Environment variable: ``TMT_PLUGIN_EXECUTE_TMT_IGNORE_DURATION``

    In plan metadata:

    .. code-block:: yaml

       ignore-duration: true|false

    On command-line:

    .. code-block:: shell

       --ignore-duration
       export TMT_PLUGIN_EXECUTE_TMT_IGNORE_DURATION=1|0


interactive
    Run tests in interactive mode, i.e. with input and output streams
    shared with tmt itself. This allows input to be passed to tests
    via stdin, e.g. responding to password prompts. Test output in this
    mode is not captured, and ``duration`` has no effect.

    Default: ``false``

    Environment variable: ``TMT_PLUGIN_EXECUTE_UPGRADE_INTERACTIVE``

    In plan metadata:

    .. code-block:: yaml

       interactive: true|false

    On command-line:

    .. code-block:: shell

       --interactive
       export TMT_PLUGIN_EXECUTE_UPGRADE_INTERACTIVE=1|0


no-progress-bar
    Disable interactive progress bar showing the current test.

    Default: ``false``

    Environment variable: ``TMT_PLUGIN_EXECUTE_UPGRADE_NO_PROGRESS_BAR``

    In plan metadata:

    .. code-block:: yaml

       no-progress-bar: true|false

    On command-line:

    .. code-block:: shell

       --no-progress-bar
       export TMT_PLUGIN_EXECUTE_UPGRADE_NO_PROGRESS_BAR=1|0


ref
    Branch, tag or commit specifying the git revision.

    Default: *not set*

    Environment variable: ``TMT_PLUGIN_EXECUTE_UPGRADE_REF``

    In plan metadata:

    .. code-block:: yaml

       ref: REVISION


    On command-line:

    .. code-block:: shell

       --ref REVISION
       export TMT_PLUGIN_EXECUTE_UPGRADE_REF=REVISION



restraint-compatible
    Run tests in the restraint-compatible mode. Enable this if
    your tests depend on the restraint scripts such as
    ``rstrnt-report-result`` or ``rstrnt-report-log``.
    Currently this option only affects whether the
    ``$OUTPUTFILE`` variable is respected, but in the future it
    will be used to enable/disable all restraint compatibility
    features.

    Default: ``false``

    Environment variable: ``TMT_PLUGIN_EXECUTE_UPGRADE_RESTRAINT_COMPATIBLE``

    In plan metadata:

    .. code-block:: yaml

       restraint-compatible: true|false

    On command-line:

    .. code-block:: shell

       --restraint-compatible
       export TMT_PLUGIN_EXECUTE_UPGRADE_RESTRAINT_COMPATIBLE=1|0


script
    Execute arbitrary shell commands and check their exit
    code which is used as a test result. The ``script`` field
    is provided to cover simple test use cases only and must
    not be combined with the :tmt:story:`/spec/plans/discover` step
    which is more suitable for more complex test scenarios.

    Default shell options are applied to the script, see
    :tmt:story:`/spec/tests/test` for more details. The default
    :tmt:story:`/spec/tests/duration` for tests defined directly
    under the execute step is ``1h``. Use the ``duration``
    attribute to modify the default limit.

    Default: *not set*

    Environment variable: ``TMT_PLUGIN_EXECUTE_UPGRADE_SCRIPT``

    In plan metadata:

    .. code-block:: yaml

       script: SCRIPT


    On command-line:

    .. code-block:: shell

       --script SCRIPT
       export TMT_PLUGIN_EXECUTE_UPGRADE_SCRIPT=SCRIPT



skip-tests-after
    If set, discovered tests would not run *after* the upgrade.

    Default: ``false``

    Environment variable: ``TMT_PLUGIN_EXECUTE_UPGRADE_SKIP_TESTS_AFTER``

    In plan metadata:

    .. code-block:: yaml

       skip-tests-after: true|false

    On command-line:

    .. code-block:: shell

       --skip-tests-after
       export TMT_PLUGIN_EXECUTE_UPGRADE_SKIP_TESTS_AFTER=1|0


skip-tests-before
    If set, discovered tests would not run *before* the upgrade.

    Default: ``false``

    Environment variable: ``TMT_PLUGIN_EXECUTE_UPGRADE_SKIP_TESTS_BEFORE``

    In plan metadata:

    .. code-block:: yaml

       skip-tests-before: true|false

    On command-line:

    .. code-block:: shell

       --skip-tests-before
       export TMT_PLUGIN_EXECUTE_UPGRADE_SKIP_TESTS_BEFORE=1|0


test
    Select tests by name.

    Default: *not set*

    Environment variable: ``TMT_PLUGIN_EXECUTE_UPGRADE_TEST``

    In plan metadata:

    .. code-block:: yaml

       test: NAMES


    On command-line:

    .. code-block:: shell

       --test NAMES
       export TMT_PLUGIN_EXECUTE_UPGRADE_TEST=NAMES



upgrade-path
    Upgrade path corresponding to a plan name in the repository with upgrade tasks.

    Default: *not set*

    Environment variable: ``TMT_PLUGIN_EXECUTE_UPGRADE_UPGRADE_PATH``

    In plan metadata:

    .. code-block:: yaml

       upgrade-path: PLAN_NAME


    On command-line:

    .. code-block:: shell

       --upgrade-path PLAN_NAME
       export TMT_PLUGIN_EXECUTE_UPGRADE_UPGRADE_PATH=PLAN_NAME



url
    URL of the git repository with upgrade tasks.

    Default: *not set*

    Environment variable: ``TMT_PLUGIN_EXECUTE_UPGRADE_URL``

    In plan metadata:

    .. code-block:: yaml

       url: REPOSITORY


    On command-line:

    .. code-block:: shell

       --url REPOSITORY
       export TMT_PLUGIN_EXECUTE_UPGRADE_URL=REPOSITORY
