HTTP

The http query allows you to fetch data from external APIs and use it within your view. It supports RESTful APIs and can extract data using JSONPath.

Example

recipes-http.yaml
apiVersion: mission-control.flanksource.com/v1
kind: View
metadata:
  name: recipes
  namespace: mc
spec:
  description: Display recipes from DummyJSON API
  cache:
    maxAge: 30m
    minAge: 5s
  display:
    title: Recipes
    icon: book
    sidebar: true
  columns:
    - name: id
      type: number
      primaryKey: true
    - name: name
      type: string
    - name: cuisine
      type: string
    - name: difficulty
      type: string
    - name: rating
      type: number
    - name: prepTimeMinutes
      type: number
    - name: cookTimeMinutes
      type: number
    - name: servings
      type: number
  queries:
    recipes:
      http:
        url: https://dummyjson.com/recipes
        jsonpath: $.recipes[:10]
 

Configuration

Field	Type	Description
url	string	The URL to send the request to.
method	string	HTTP method (GET, POST, PUT, DELETE, PATCH). Default: GET.
body	string	Request body for POST/PUT/PATCH requests. Supports templating.
jsonpath	string	JSONPath expression to extract specific data from the response.
headers	list	List of headers to include in the request.
connection	string	Reference to a connection for authentication.
username	EnvVar	Basic auth username.
password	EnvVar	Basic auth password.
bearer	EnvVar	Bearer token.

JSONPath Extraction

If the API returns a nested structure, you can use jsonpath to extract the list of items you want to treat as rows.

For example, if the API returns:

{
  "status": "success",
  "data": {
    "items": [
      { "id": 1, "name": "A" },
      { "id": 2, "name": "B" }
    ]
  }
}

Use jsonpath: "$.data.items" to extract the array.

Authentication

You can provide authentication details inline or reference a stored connection.

Inline Basic Auth:

http:
  url: https://api.example.com/data
  username:
    value: myuser
  password:
    value: mypass

Using a Connection:

http:
  url: https://api.example.com/data
  connection: connection://my-api-creds

Connection Field

The connection field references a stored Connection resource. When specified, Mission Control hydrates the connection by:

1. Looking up the connection by name in the view's namespace
2. Extracting authentication details (username, password, bearer token, OAuth credentials)
3. Applying TLS configuration (CA, cert, key, insecure skip verify)
4. Merging custom headers defined in the connection
5. Resolving any environment variable references in connection properties

Connection properties that can be inherited:

• url - Base URL for the API
• username / password - HTTP Basic Authentication
• bearer - Bearer token for token-based auth
• oauth - OAuth2 credentials (clientID, clientSecret, tokenURL, scopes, params)
• tls - TLS configuration (ca, cert, key, insecureSkipVerify)
• headers - Additional HTTP headers

Example with connection:

http-connection.yaml
apiVersion: mission-control.flanksource.com/v1
kind: View
metadata:
  name: http-connection-example
  namespace: default
spec:
  description: Example of using a stored connection for HTTP queries
  display:
    title: GitHub Repos
    icon: github
    sidebar: true
  columns:
    - name: name
      type: string
      primaryKey: true
    - name: stars
      type: number
    - name: language
      type: string
  queries:
    repos:
      http:
        url: https://api.github.com/users/flanksource/repos
        headers:
          - name: Accept
            value: application/vnd.github.v3+json
  mapping:
    name: row.name
    stars: row.stargazers_count
    language: row.language

Templating Support

The body field supports Go templating, allowing dynamic request payloads:

http-post-body.yaml
apiVersion: mission-control.flanksource.com/v1
kind: View
metadata:
  name: http-post-body-example
  namespace: default
spec:
  description: Example of POST request with templated body using httpbin
  display:
    title: HTTP POST Echo
    icon: web
  columns:
    - name: url
      type: string
      primaryKey: true
    - name: origin
      type: string
    - name: content_type
      type: string
    - name: sent_message
      type: string
      description: The message sent in the POST body (echoed back by httpbin)
  templating:
    - key: message
      label: Message
      values:
        - "Hello from Mission Control"
        - "Test message"
        - "API call"
      default: "Hello from Mission Control"
  queries:
    echo:
      http:
        url: https://httpbin.org/post
        method: POST
        body: |
          {"message": "$(var.message)", "timestamp": "2024-01-15T10:30:00Z"}
        headers:
          - name: Content-Type
            value: application/json
  mapping:
    url: row.url
    origin: row.origin
    content_type: row.headers["Content-Type"]
    sent_message: row.json.message

Template variables available in the body:

• $(var.<name>) - View template variables
• Environment variables via $(ENV_VAR_NAME)

Response Size Limits

By default, HTTP responses are limited to 25 MB to prevent memory issues with large payloads. You can configure this limit using the view.http.body.max_size_bytes property.

Configuring the Limit

Set the property via environment variable or configuration:

# Increase limit to 50 MB
properties:
  view.http.body.max_size_bytes: '52428800'

Size Limit Error Handling

When a response exceeds the configured limit, the view returns an error with details:

http response body size (32 MB) exceeds maximum allowed (25 MB); increase limit via property "view.http.body.max_size_bytes"

To resolve:

1. Increase the limit via the property (if you have sufficient memory)
2. Use pagination in the API request to fetch smaller chunks
3. Apply filters in the API request to reduce response size
4. Use jsonpath to extract only the needed subset of data

Example with pagination:

queries:
  paged-data:
    http:
      url: https://api.example.com/items?page=1&per_page=100
      connection: connection://my-api