HTTP

The HTTP check sends HTTP requests and evaluates the response.

apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: http-single
  labels:
    canary: http
spec:
  schedule: "@every 5m"
  http:
    - name: http pass response 200 status code
      url: https://httpbin.flanksource.com/status/200
      thresholdMillis: 3000
      responseCodes: [200]
      maxSSLExpiry: 7

Field	Description	Scheme
`name*`	Name of the check, must be unique within the canary	`string`
`awsSigV4`	AWS Signature Version 4 signing configuration	AWSSigV4
`body`	Request Body Contents	`string`
`connection`	Connection name e.g. connection://http/google	`string`
`connection`	Connection reference or URL to use. Inline username and password can supply credentials when the check supports them	Connection
`crawl`	Crawl configuration for following links	Crawl
`digest`	Authenticate using HTTP Digest authentication	`boolean`
`endpoint`	Deprecated: Use url instead	`string`
`env`	Environment variables that are accessible while templating	[]EnvVar
`headers`	Header fields	[]EnvVar
`maxRedirects`	Maximum number of redirects to follow. Defaults to `10`	`integer`
`maxSSLExpiry`	Maximum number of days until the SSL Certificate expires Equivalent to `test.expr: sslAge < Age("7d")`	`integer`
`method`	HTTP Request method. Defaults to `GET`	`string`
`ntlm`	When set to true, authenticates using NTLM v1 protocol	`boolean`
`ntlmv2`	When set to true, authenticates using NTLM v2 protocol	`boolean`
`oauth2`	OAuth2 configuration	OAuth2Config
`password`	Password for authentication	EnvVar
`password`		EnvVar
`responseCodes`	Expected HTTP response codes Equivalent to `test.expr: code in [200, 201]`	[]int
`responseContent`	Expected response body content Equivalent to `test.expr: content.contains("value")`	`string`
`responseJSONContent`	Deprecated and no-op. Use `test.expr` with `json` or `jsonpath` instead.	JSONCheck
`templateBody`	If true body is templated	`boolean`
`thresholdMillis`	Maximum duration in milliseconds for the HTTP request. When unset, the HTTP client timeout is used.	`integer`
`tlsConfig`	TLS config	TLSConfig
`url`	HTTP URL. Required unless `connection` supplies the URL	`string`
`username`	Username for authentication	EnvVar
`username`		EnvVar
`dependsOn`	Checks that must complete before this check runs	`[]string`
`description`	Description for the check	`string`
`display`	Expression to change the formatting of the display	`Expression`
`icon`	Icon for overwriting default icon on the dashboard	Icon
`labels`	Labels for check	`[map[string]string]`
`markFailOnEmpty`	If a transformation or datasource returns empty results, the check should fail	`boolean`
`metrics`	Metrics to export from	`[]Metrics`
`namespace`	Namespace to insert the check result into when it differs from the canary namespace	`string`
`relationships`	Relationships that link check results to components or configs	Relationship
`test`	Evaluate whether a check is healthy	`Expression`
`transform`	Transform data from a check into multiple individual checks	`Expression`
`transformDeleteStrategy`	Status to apply to transformed checks when the source check no longer returns them	`string`

TLS Config

Field	Description	Scheme
`ca`	PEM encoded certificate of the CA to verify the server certificate	EnvVar
`cert`	PEM encoded client certificate	EnvVar
`handshakeTimeout`	Timeout for SSL handshake (defaults to 10 seconds)	Duration
`insecureSkipVerify`	Controls whether a client verifies the server's certificate chain and host name	`boolean`
`key`	PEM encoded client private key	EnvVar

Verifying against an external CA

apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: http-check
spec:
  schedule: "@every 30s"
  http:
    - name: http pass response 200 status code
      url: https://httpbin.flanksource.com/status/200
      tlsConfig:
        ca:
          valueFrom:
            secretKeyRef:
              name: ca-cert
              key: ca.pem

OAuth

Field	Description	Scheme
`params`	Parameters for OAuth	`[map[string]string]`
`scope`	Scope for OAuth token request	`[]string`
`tokenURL`	URL for OAuth Token	`string`

AWS Signature Version 4

Use awsSigV4 to sign requests with AWS Signature Version 4.

Field	Description	Scheme
`accessKey`	Access Key ID	EnvVar
`assumeRole`	Role ARN to assume	`string`
`connection`	The connection url to use, mutually exclusive with `accessKey` and `secretKey`	Connection
`endpoint`	Custom AWS Endpoint to use	`string`
`region`	The AWS region	`string`
`secretKey`	Secret Access Key	EnvVar
`service`	AWS service name for request signing	`string`
`sessionToken`	Session token	EnvVar
`skipTLSVerify`	Skip TLS verify when connecting to AWS	`boolean`

http-aws-sigv4.yaml
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: http-aws-sigv4
spec:
  http:
    - name: aws-sigv4-signing
      url: https://httpbin.flanksource.com/headers
      awsSigV4:
        accessKey:
          value: test-key
        secretKey:
          value: test-secret
        region: us-east-1
        service: execute-api
      test:
        expr: 'json.headers.Authorization.startsWith("AWS4-HMAC-SHA256")'

Crawl Configuration

Field	Description	Scheme
`allowedDomains`	List of domains to crawl	`[]string`
`allowedURLFilters`	List of URL regex patterns to include	`[]string`
`delay`	Duration to wait before creating a new request to the matching domains. Defaults to `500ms`	Duration
`depth`	Maximum number of links to follow	`integer`
`disallowedDomains`	List of domains to exclude from crawling	`[]string`
`disallowedURLFilters`	List of URL regex patterns to exclude	`[]string`
`parallelism`	Number of the maximum allowed concurrent requests. Defaults to `2`	`integer`
`randomDelay`	Extra randomized duration to wait added to Delay before creating a new request. Defaults to `100ms`	Duration

http-crawl.yaml
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: http-crawler
spec:
  schedule: "@daily"
  http:
    - name: docs
      url: https://httpbin.flanksource.com/
      display:
        expr: missing.join("\n")
      crawl:
        parallelism: 2
        delay: 100ms
        randomDelay: 50ms
        depth: 2

Result Variables

Result variables can be used in test, display and transform expressions

Name	Description	Scheme
`code`	HTTP response code	`int`
`headers`	HTTP response headers	`map[string]string`
`elapsed`	HTTP Request duration	`time.Duration`
`sslAge`	Time until SSL certificate expires	`time.Duration`
`content`	HTTP Response body	string
`json`	If `Content-Type=application/json` response body converted into JSON object	`JSON`

The above canary (http-check.yaml) is functionally equivalent to http-check-expr.yaml below

apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: http-tls-duration
spec:
  schedule: "@every 5m"
  http:
    - name: http pass response 200 status code
      endpoint: https://httpbin.flanksource.com/status/200
      test:
        expr: "code in [200,201,301] && sslAge > Duration('7d')"

Authentication

Basic Authentication

http_auth.yaml
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: http-basic-auth
spec:
  schedule: "@every 1m"
  http:
    - name: "basic auth fail"
      url: https://httpbin.flanksource.com/basic-auth/hello/world
      responseCodes: [401]
    - name: "basic auth pass"
      url: https://httpbin.flanksource.com/basic-auth/hello/world
      responseCodes: [200]
      username:
        valueFrom:
          secretKeyRef:
            name: httpbin-secret
            key: username
      password:
        valueFrom:
          secretKeyRef:
            name: httpbin-secret
            key: password

Template Body Variables

Name	Description	Scheme
`canary.name`	Canary name	`string`
`canary.namespace`	Canary namespace	`string`
`canary.labels`	Canary labels	`map[string]string`
`canary.id`	Persisted Canary ID	`string`
`check.name`	Check name	`string`
`check.id`	Check ID	`string`
`check.description`	Check description	`string`
`check.labels`	Check labels	`map[string]string`
`check.endpoint`	Check endpoint/URL	`string`
`{envVar.name}`	Environment variables defined in `env`	`any`

Variables defined in env are available to template with the name that's configured on the spec. Eg: In the following spec, the var db, defined in env, is available as {{.db}} during templating.

Dependency Output Variables

When an HTTP check has dependsOn configured, the outputs variable is available for templating. It contains the results from all dependency checks that have completed successfully.

Name	Description	Scheme
`outputs.<checkName>.pass`	Whether the dependency check passed	`bool`
`outputs.<checkName>.message`	Result message from the dependency check	`string`
`outputs.<checkName>.duration`	Duration of the dependency check in milliseconds	`int64`
`outputs.<checkName>.code`	HTTP response code from the dependency check	`int`
`outputs.<checkName>.headers`	Response headers from the dependency check	`map[string]string`
`outputs.<checkName>.content`	Raw response body from the dependency check	`string`
`outputs.<checkName>.json`	Parsed JSON response from the dependency check (if `Content-Type=application/json`)	`JSON`
`outputs.<checkName>.elapsed`	HTTP request duration from the dependency check	`time.Duration`
`outputs.<checkName>.sslAge`	Time until SSL certificate expires from the dependency check	`time.Duration`

See Request Chaining for examples.

Templating request body from env variables

http_template.yaml
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: templated-http
spec:
  schedule: "@every 5m"
  http:
    - name: templated-http
      url: https://webhook.site/bf6a337a-4413-4839-9866-1d28ed96d65f
      responseCodes: [200]
      templateBody: true
      method: POST
      env:
        - name: db
          valueFrom:
            secretKeyRef:
              name: incident-commander-postgres
              key: POSTGRES_HOST
      body: |
        {
          "canary": "{{.canary.name}}",
          "secret": "{{.db}}"
        }

Templating URL from env variables

http_user_pass_template.yaml
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: http-basic-auth-url
spec:
  http:
    - name: test-url-via-env
      # The URL can be templated from arbritrary values using the env field and $(.) syntax
      url: $(.url)
      env:
        - name: url
          value: https://hello:world2@httpbin.flanksource.com/basic-auth/hello/world2
    - name: test-basic-via-env
      # the url can be constructed from multiple variables
      url: https://$(.user):$(.pass)@httpbin.flanksource.com/basic-auth/hello/world
      templateBody: true
      body: |
        {{. | toJSONPretty " " }}
      responseCodes: [200]
      env:
        - name: user
          value: hello
        - name: pass
          value: world

See Escaping variables

Metrics

HTTP Check Metrics
`canary_check_http_response_status{status, statusClass, url}`	Counter	Response code counter for each endpoint
`canary_check_http_ssl_expiry{url}`	Gauge

Status class is one of 1xx, 2xx, 3xx, 4xx, 5xx

Adding custom metrics

metrics.yaml
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: http-pass-single
spec:
  schedule: "@every 5m"
  http:
    - name: http-minimal-check
      url: https://httpbin.flanksource.com/status/200
      metrics:
        - name: httpbin_count
          type: counter
          value: "1"
          labels:
            - name: check_name
              valueExpr: check.name
            - name: code
              valueExpr: code
        - name: httpbin_2xx_duration
          type: counter
          value: elapsed.getMilliseconds()
          labels:
            - name: check_name
              valueExpr: check.name

Transforming metrics into individual checks

metrics.yaml
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: exchange-rates
spec:
  schedule: "every 30 @minutes"
  http:
    - name: exchange-rates
      url: https://api.frankfurter.app/latest?from=USD&to=GBP,EUR,ILS
      transform:
        expr: |
          {
            'metrics': json.rates.keys().map(k,  {
              'name': "exchange_rate",
              'type': "gauge",
              'value': json.rates[k],
              'labels': {
                "from": json.base,
                "to": k
              }
            })
          }.toJSON()
      metrics:
        - name: exchange_rate_api
          type: histogram
          value: elapsed.getMilliseconds()

TLS Config​

OAuth​

AWS Signature Version 4​

Crawl Configuration​

Result Variables​

Authentication​

Template Body Variables​

Dependency Output Variables​

Metrics​