Publishing test results
You can publish test results by using other continuous integration and continuous delivery (CI/CD) tools with the IBM Cloud® command line interface (CLI) to integrate with IBM Cloud® DevOps Insights. Test results inform DevOps Insights about the tests that are run during the build process. These tests must generate some kind of JSON or XML-based file.
The generated test file is published as a test record. Along with the test file, this record contains the application name, build number, types of test, and other fields. The application name and build ID in the test record must match the ones that are used in the build record for a specific build. After you run a test in your CI/CD tool, you can upload the results to DevOps Insights
Before you begin
Before you publish test results, you must publish a deployment record. For more information, see Publishing a deployment record by using other CI/CD tools.
Publishing test results
Use the testrecord-publish command to upload a test record. The following example script uploads functional verification test (FVT) results to DevOps Insights:
# Run tests and generate a test results file here.
#install the DevOps Insights plugin
ibmcloud plugin install -f doi
# Log in to IBM Cloud if you are not already logged in. Assumes that $API_KEY environment variable has been set as a secured property
ibmcloud login --apikey $API_KEY --no-region
ibmcloud doi testrecord-publish --logicalappname="$MY_APP_NAME" --buildnumber="$MY_BUILD_NUMBER" --filelocation fvttest.json --type fvt
In the example script, the testrecord-publish command specifies that the command uploads test records. The --filelocation flag indicates the location of the test results file relative to the root directory of the job.
The --type flag indicates the type of test result.
The following example script runs tests and then uploads Mocha results to DevOps Insights:
# Run tests and generate a test results file here.
istanbul cover --report lcov --report json-summary _mocha -- -R xunit -O output=./test/results/mocha.xml 'test/**/*.js'
#install the DevOps Insights plugin
ibmcloud plugin install -f doi
# Log in to IBM Cloud if you are not already logged in. Assumes that $API_KEY environment variable has been set as a secured property
ibmcloud login --apikey $API_KEY --no-region
#Publish test results. Assumes that MY_APP_NAME and MY_BUILD_NUMBER environment variables are already set
ibmcloud doi testrecord-publish --logicalappname="$MY_APP_NAME" --buildnumber="$MY_BUILD_NUMBER" --filelocation /test/results/mocha.xml --type unittest
ibmcloud doi testrecord-publish --logicalappname="$MY_APP_NAME" --buildnumber="$MY_BUILD_NUMBER" --filelocation ./coverage/coverage-summary.json --type code
The command supports the following type values:
| Type | Description |
|---|---|
unittest |
Unit test results |
fvt |
Functional verification test results |
code |
Code coverage results |
sonarqube |
SonarQube scan results |
staticsecurityscan |
Static security scan results from IBM Application Security on Cloud |
dynamicsecurityscan |
Dynamic security scan results from IBM Application Security on Cloud |
Test records must provide data in one of the following supported formats:
| Test Type | Supported Formats |
|---|---|
| Functional verification test | Mocha, xUnit, Karma/Mocha |
| Unit test | Mocha, xUnit, Karma/Mocha |
| Code coverage | Cobertura, lcov, JaCoCo |
| Static AppScan | Static App Scans that are provided by IBM Application Security on Cloud |
| Dynamic AppScan | Dynamic App Scans that are provided by IBM Application Security on Cloud |
| SonarQube | Scan data that is provided by SonarQube scans |
Viewing test results
When your CI/CD tool runs, it publishes the test result data to DevOps Insights. To view the test result data in DevOps Insights, go to the Quality Dashboard page. For more information, see DevOps data aggregation.
Next steps
Evaluate gates by using other CI/CD tools.