JUnit

Use a JUnit check to run a Kubernetes pod and ingest JUnit XML reports from testResults.

junit.yaml
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: junit-pass
spec:
  schedule: "@every 2h"
  junit:
    - testResults: "/tmp/junit-results/"
      name: junit-pass
      test:
        expr: results.failed == 0 && results.passed > 0
      display:
        expr: "string(results.failed) + ' of ' +  string(results.passed)"
      spec:
        containers:
          - name: jes
            image: docker.io/tarun18/junit-test-pass
            command: ["/start.sh"]

Field	Description	Scheme
name*	Name of the check, must be unique within the canary	string
spec*	Pod specification	Spec
testResults*	Path to JUnit test results	string
artifacts	Specify what files/folders created in the pod need to be archived	[]Artifact
timeout	Timeout in minutes to wait for specified container to finish its job. Defaults to 5	integer
dependsOn	Checks that must complete before this check runs	[]string
description	Description for the check	string
display	Expression to change the formatting of the display	Expression
icon	Icon for overwriting default icon on the dashboard	Icon
labels	Labels for check	[map[string]string]
markFailOnEmpty	If a transformation or datasource returns empty results, the check should fail	boolean
metrics	Metrics to export from	[]Metrics
namespace	Namespace to insert the check result into when it differs from the canary namespace	string
relationships	Relationships that link check results to components or configs	Relationship
test	Evaluate whether a check is healthy	Expression
transform	Transform data from a check into multiple individual checks	Expression
transformDeleteStrategy	Status to apply to transformed checks when the source check no longer returns them	string

Export custom metrics from JUnit results

junit.yaml
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: junit-metrics
  labels:
    part-of: canary-tools
spec:
  schedule: "@every 2h"
  severity: high
  junit:
    - testResults: "/tmp/junit-results/"
      name: junit-pass
      test:
        expr: results.failed == 0 && results.passed > 0
      display:
        expr: "string(results.failed) + ' of ' +  string(results.passed)"
      spec:
        containers:
          - name: jes
            image: docker.io/tarun18/junit-test-pass
            command: ["/start.sh"]
      metrics:
        - name: junit_check_pass_count
          type: gauge
          value: results.passed
          labels:
            - name: suite_name
              valueExpr: results.suites[0].name
        - name: junit_check_failed_count
          type: gauge
          value: results.failed
          labels:
            - name: part_of
              valueExpr: canary.labels['part-of']
        - name: junit_check_duration_ms
          type: histogram
          value: results.duration * 1000.0
          labels:
            - name: suite_name
              valueExpr: results.suites[0].name

Troubleshooting

1. Add the skipDelete annotation to the canary to prevent created pods from being deleted.

Test Result Variables

Name	Description	Scheme
suites		[]JunitTestSuite
passed	Number of passing tests	int
failed	Number of failed tests	int
skipped	Number of skipped tests	int
error	Number of errors produced when running the tests	int
duration	Total time in seconds	float64

JUnit Test Suite

Name	Description	Scheme
name		string
tests		[]JunitTest
passed	Number of passing tests	int
failed	Number of failed tests	int
skipped	Number of skipped tests	int
error	Number of errors produced when running the tests	int
duration	Total time in seconds	float64

JUnit Test

Name	Description	Scheme
name		string
classname	an additional descriptor for the hierarchy of the test.	string
duration	Time in seconds	float64
status	One of passed, skipped, failed or error	string
message	Text included with a skipped, failed, or errored test case	string
properties	Additional info about the test	map[string]string
error	Any errors encountered when running a test	string
stdout	Standard output produced during test	string
stderr	Standard error output produced during test	string

Artifact

Field	Description	Type	Required
path	Path or glob.	string	true

See artifacts