# Recording test results with the Launchable CLI

{% hint style="info" %}
Before you start recording test results to test sessions, consider your [#test-session-layouts](../../../concepts/test-session.md#test-session-layouts "mention").
{% endhint %}

Launchable uses your test results from each [test-session.md](../../../concepts/test-session.md "mention") to provide features.

To record tests to a test session, after running tests, point the CLI to your test report files to collect test results for the build.

Launchable uses the `<BUILD NAME>` value from [recording-builds-with-the-launchable-cli](../recording-builds-with-the-launchable-cli/ "mention") to connect the test results with the changes in the build. You need to record a build before you can record test results.

The general structure of this command is:

```bash
launchable record tests --build <BUILD NAME> <TOOL NAME> <PATHS TO REPORT FILES>
```

This command varies slightly based on the test runner/build tool you use. See below for specific instructions.

After recording tests, you can see your recorded test sessions them on the **Test Sessions** page of the Launchable dashboard at [app.launchableinc.com](https://app.launchableinc.com/). You can click into each session to view details and failed tests.

<figure><img src="../../../.gitbook/assets/Test Sessions 20221202.png" alt=""><figcaption></figcaption></figure>

The CLI will also output a link to view each session's details in the dashboard.

## Instructions for test runners/build tools

### Android Debug Bridge (adb)

Currently, the CLI doesn't have a `record tests` command for ADB. Use the command for [#gradle](./#gradle "mention") instead.

### Ant

After running tests, point the CLI to your test report file(s):

```bash
launchable record tests --build <BUILD NAME> ant <PATH TO JUNIT XML>
```

Then, follow the instructions for [ensuring-record-tests-always-runs.md](ensuring-record-tests-always-runs.md "mention").

### Bazel

After running tests, point the CLI to your test report files to record test results:

```bash
launchable record tests --build <BUILD NAME> bazel .
```

Then, follow the instructions for [ensuring-record-tests-always-runs.md](ensuring-record-tests-always-runs.md "mention").

### Behave

First, in order to generate reports that Launchable can consume, add the `--junit` option to your existing `behave` command:

```bash
# run the tests however you normally do
behave --junit
```

Then, after running tests, point the CLI to your test report files to record test results:

```bash
launchable record tests --build <BUILD NAME> behave ./reports/*.xml
```

Then, follow the instructions for [ensuring-record-tests-always-runs.md](ensuring-record-tests-always-runs.md "mention").

### CTest

First, run your tests with `ctest -T test --no-compress-output`. These options ensure test results are written to the `Testing` directory.

Then, after running tests, point the CLI to your test report files to record test results:

```bash
launchable record tests --build <BUILD NAME> ctest Testing/**/Test.xml
```

Then, follow the instructions for [ensuring-record-tests-always-runs.md](ensuring-record-tests-always-runs.md "mention").

### cucumber

Launchable supports JSON and JUnit XML reports generated by cucumber (using `-f junit` or `-f json`. Follow the instructions for whichever one you use. JUnit XML is the default.

{% tabs %}
{% tab title="JUnit XML" %}
If you run cucumber with the `-f junit` option, like this:

```bash
bundle exec cucumber -f junit -o reports
```

After running tests, point the CLI to your test report files to record test results:

```
launchable record tests --build <BUILD NAME> cucumber ./reports/**/*.xml
```

Then, follow the instructions for [ensuring-record-tests-always-runs.md](ensuring-record-tests-always-runs.md "mention").
{% endtab %}

{% tab title="JSON" %}
If you run cucumber with the `-f json` option, like this:

```bash
bundle exec cucumber -f json -o reports
```

After running tests, point the CLI to your test report files to record test results (note the `--json` option):

```
launchable record tests --build <BUILD NAME> cucumber --json ./reports/**/*.json
```

Then, follow the instructions for [ensuring-record-tests-always-runs.md](ensuring-record-tests-always-runs.md "mention").
{% endtab %}
{% endtabs %}

Note: If you receive a warning messages such as `Cannot find test file Test-feature-example.xml`, set the project's root directory path with the `--base` option:

{% code overflow="wrap" %}
```
launchable record tests --build <BUILD NAME> --base /example/project cucumber /example/project/reports/**/*.xml
```
{% endcode %}

### Cypress

After running tests, point the CLI to your test report files to record test results:

```bash
launchable record tests --build <BUILD NAME> cypress ./report/*.xml
```

Then, follow the instructions for [ensuring-record-tests-always-runs.md](ensuring-record-tests-always-runs.md "mention").

### GoogleTest

First, configure GoogleTest to produce JUnit compatible report files. See [their documentation](https://github.com/google/googletest/blob/master/docs/advanced.md#generating-an-xml-report) for how to do this. You'll end up with a command something like this:

```bash
# run the tests however you normally do
./my-test --gtest_output=xml:./report/my-test.xml
```

Then, after running tests, point the CLI to your test report files to record test results:

```bash
launchable record tests --build <BUILD NAME> googletest ./report
```

Then, follow the instructions for [ensuring-record-tests-always-runs.md](ensuring-record-tests-always-runs.md "mention").

### Go Test

First, in order to generate reports that Launchable can consume, use [go-junit-report](https://github.com/jstemmer/go-junit-report) to generate a JUnit XML file after you run tests:

{% code overflow="wrap" %}
```bash
# install JUnit report formatter
go get -u github.com/jstemmer/go-junit-report

# run the tests however you normally do, then produce a JUnit XML file
go test -v ./... | go-junit-report -set-exit-code > report.xml
```
{% endcode %}

Then point the CLI to your test report file to record test results:

```bash
launchable record tests --build <BUILD NAME> go-test report.xml
```

Then, follow the instructions for [ensuring-record-tests-always-runs.md](ensuring-record-tests-always-runs.md "mention").

### Gradle

After running tests, point the CLI to your test report files to record test results:

{% code overflow="wrap" %}
```bash
launchable record tests --build <BUILD NAME> gradle ./build/test-results/test/
```
{% endcode %}

Then, follow the instructions for [ensuring-record-tests-always-runs.md](ensuring-record-tests-always-runs.md "mention").

Notes:

* By default, Gradle's report files are saved to `build/test-results/test/`, but that might be different depending on how your Gradle project is configured.
* You can specify multiple directories if you do multi-project build.
* For a large project, a dedicated Gradle task to list up all report directories might be convenient. See [the upstream documentation](https://docs.gradle.org/current/userguide/java\_testing.html#test\_reporting) for more details and insights.
* Alternatively, you can specify a glob pattern for directories or individual test report files (this pattern might already be specified in your pipeline script for easy copy-pasting), e.g. `gradle **/build/**/TEST-*.xml`.

### Jest

First, in order to generate reports that Launchable can consume, use [jest-junit](https://www.npmjs.com/package/jest-junit) to generate a JUnit XML file after you run tests.

{% code overflow="wrap" %}
```bash
# install jest-junit reporter
npm install jest-junit --save-dev
# or
yarn add --dev jest-junit
```
{% endcode %}

You'll need to configure jest-junit to include file paths in reports.

You can do this using environment variables:

{% tabs %}
{% tab title="Using environment variables" %}
Recommended config:

```bash
export JEST_JUNIT_CLASSNAME="{classname}"
export JEST_JUNIT_TITLE="{title}"
export JEST_JUNIT_SUITE_NAME="{filepath}"
```

Minimum config:

```bash
export JEST_JUNIT_SUITE_NAME="{filepath}"
```
{% endtab %}

{% tab title="Using package.json" %}
Add the following lines to your `package.json`. The detail is the [jest-junit](https://www.npmjs.com/package/jest-junit) section.

Recommended config:

```json
// package.json
"jest-junit": {
  "suiteNameTemplate": "{filepath}",
  "classNameTemplate": "{classname}",
  "titleTemplate": "{title}"
}
```

Minimum config:

```json
// package.json
"jest-junit": {
  "suiteNameTemplate": "{filepath}"
}
```
{% endtab %}
{% endtabs %}

Then, run `jest` using jest-junit:

```bash
# run tests with jest-junit
jest --ci --reporters=default --reporters=jest-junit
```

Then, after running tests, point the CLI to your test report files to record test results:

```bash
# send test results to Launchable
launchable record tests --build <BUILD NAME> jest your-junit.xml
```

Finally, follow the instructions for [ensuring-record-tests-always-runs.md](ensuring-record-tests-always-runs.md "mention").

### Maven

{% hint style="info" %}
Launchable supports test reports generated using [Surefire](https://maven.apache.org/surefire/maven-surefire-plugin/), the default report plugin for [Maven](https://maven.apache.org).
{% endhint %}

After running tests, point the CLI to your test report files to record test results:

{% code overflow="wrap" %}
```bash
launchable record tests --build <BUILD NAME> maven './**/target/surefire-reports'
```
{% endcode %}

_Note: The invocation above relies on the CLI to expand GLOBs like `**`._

Then, follow the instructions for [ensuring-record-tests-always-runs.md](ensuring-record-tests-always-runs.md "mention").

### minitest

First, use [minitest-ci](https://github.com/circleci/minitest-ci) to output test results to a file. If you already store your test results on your CI server, it may already be installed.

Then, after running tests, point the CLI to your test report files to record test results:

{% code overflow="wrap" %}
```bash
launchable record tests --build <BUILD NAME> minitest "$CIRCLE_TEST_REPORTS/reports"
```
{% endcode %}

Then, follow the instructions for [ensuring-record-tests-always-runs.md](ensuring-record-tests-always-runs.md "mention").

### NUnit Console Runner

{% hint style="info" %}
Launchable CLI accepts [NUnit3 style test report XML files](https://docs.nunit.org/articles/nunit/technical-notes/usage/XML-Formats.html) produced by NUnit.
{% endhint %}

After running tests, point the CLI to your test report file(s) to record test results:

{% code overflow="wrap" %}
```bash
launchable record tests --build <BUILD NAME> nunit path/to/TestResult.xml
```
{% endcode %}

Then, follow the instructions for [ensuring-record-tests-always-runs.md](ensuring-record-tests-always-runs.md "mention").

### pytest

Launchable supports JSON and JUnit XML reports generated by pytest (using `--junit-xml` or `--report-log`). Follow the instructions for whichever one you use. JUnit XML is the default.

{% tabs %}
{% tab title="JUnit XML" %}
First, run tests with the `--junit-xml` option:

<pre class="language-bash"><code class="lang-bash"><strong>pytest --junit-xml=test-results/results.xml
</strong></code></pre>

{% hint style="warning" %}
pytest changed its default test report format from `xunit1` to `xunit2` in version 6. Unfortunately, the new `xunit2` format does not include file paths, which Launchable needs.

Thefore, if you are using pytest 6 or newer, you must also specify `junit_family=legacy` as the report format. See [Deprecations and Removals — pytest documentation](https://docs.pytest.org/en/latest/deprecations.html#junit-family-default-value-change-to-xunit2) for instructions.
{% endhint %}

Then, after running tests, point the CLI to your test report file(s) to record test results (note the `--json` option):

```bash
launchable record tests --build <BUILD NAME> pytest ./test-results/
```

Then, follow the instructions for [ensuring-record-tests-always-runs.md](ensuring-record-tests-always-runs.md "mention").
{% endtab %}

{% tab title="pytest-reportlog JSON" %}
If you use the [pytest-dev/pytest-reportlog](https://github.com/pytest-dev/pytest-reportlog) plugin to generate reports, like this:

```
pytest --report-log=test-results/results.json
```

After running tests, point the CLI to your test report file(s) to record test results:

```
launchable record tests --build <BUILD NAME> pytest --json ./tests-results/
```

Then, follow the instructions for [ensuring-record-tests-always-runs.md](ensuring-record-tests-always-runs.md "mention").
{% endtab %}
{% endtabs %}

### Robot

After running tests with `robot`, point the CLI to your test report files to record test results:

```bash
launchable record tests --build <BUILD NAME> robot output.xml
```

Then, follow the instructions for [ensuring-record-tests-always-runs.md](ensuring-record-tests-always-runs.md "mention").

### RSpec

First, use [rspec\_junit\_formatter](https://github.com/sj26/rspec\_junit\_formatter) to output test results to a file in RSpec. If you already have a CI server storing your test results it may already be installed:

```bash
bundle exec rspec --format RspecJunitFormatter --out report/rspec.xml
```

Then, after running tests, point the CLI to your test report files to record test results:

```bash
launchable record tests --build <BUILD NAME> rspec ./report
```

Then, follow the instructions for [ensuring-record-tests-always-runs.md](ensuring-record-tests-always-runs.md "mention").

## Other instructions

If you're not using any of these, see [raw.md](../../../resources/integrations/raw.md "mention") or[using-the-generic-file-based-runner-integration.md](../../../resources/integrations/using-the-generic-file-based-runner-integration.md "mention").
