Logs Action

The Logs action fetches and queries logs from various backends. Use it to retrieve logs for debugging, analysis, or to pass to downstream actions like AI analysis.

fetch-pod-logs.yaml
apiVersion: mission-control.flanksource.com/v1
kind: Playbook
metadata:
  name: analyze-pod-errors
spec:
  on:
    config:
      - types:
          - Kubernetes::Pod
  actions:
    - name: fetch-logs
      logs:
        kubernetes:
          namespace: '{{.config.tags.namespace}}'
          pod: '{{.config.name}}'
          since: 1h
          tailLines: 500
    - name: analyze
      ai:
        prompt: |
          Analyze these logs and identify any errors or issues:
          {{.actions.fetch-logs.result.logs | toJSON}}

Field	Description	Scheme
name*	Step Name	string
logs	Logs Action	Logs
delay	A delay before running the action e.g. 8h	Duration or CEL with Playbook Context
filter	Conditionally run an action	CEL with Playbook Context
runsOn	Which runner (agent) to run the action on	[]Agent
templatesOn	Where templating (and secret management) of actions should occur	host or agent
timeout	Timeout on this action.	Duration

Logs

Field	Description	Scheme
cloudwatch	Query logs from AWS CloudWatch Logs	CloudWatch
kubernetes	Fetch logs directly from Kubernetes pods	Kubernetes
loki	Query logs from Grafana Loki	Loki
opensearch	Query logs from OpenSearch/Elasticsearch	OpenSearch
dedupe.fields	Fields to use for identifying duplicate log entries (e.g., ['message', 'level'])	[]string
dedupe.window	Time window for deduplication (e.g., 5m, 1h). Logs with identical dedupe fields within this window are merged	Duration
mapping.id	Source field names for unique log identifier	[]string
mapping.message	Source field names for log message content	[]string
mapping.severity	Source field names for log level/severity	[]string
mapping.timestamp	Source field names for timestamp (tries each until non-empty)	[]string
match	CEL expressions to filter logs after retrieval. Only logs matching all expressions are returned	[]string

CloudWatch

Query logs from AWS CloudWatch Logs.

Field	Description	Scheme
logGroup*	CloudWatch log group name	string
accessKey	AWS access key ID	EnvVar
connection	AWS connection for credentials	AWSConnection
endTime	End time - relative or Unix timestamp. Defaults to now	string
filterPattern	CloudWatch filter pattern (e.g., ERROR or { $.level = "error" })	string
limit	Maximum number of events to return	integer
logStream	Specific log stream name. If omitted, searches all streams	string
region	AWS region (e.g., us-east-1)	string
secretKey	AWS secret access key	EnvVar
startTime	Start time - relative (e.g., 1h) or Unix timestamp in ms	string

Example

cloudwatch-lambda-logs.yaml
apiVersion: mission-control.flanksource.com/v1
kind: Playbook
metadata:
  name: fetch-lambda-errors
spec:
  parameters:
    - name: function_name
      label: Lambda Function
  actions:
    - name: get-logs
      logs:
        cloudwatch:
          connection: connection://aws/production
          region: us-east-1
          logGroup: '/aws/lambda/{{.params.function_name}}'
          filterPattern: 'ERROR'
          startTime: 6h
          limit: 200

Kubernetes

Fetch logs directly from Kubernetes pods using the Kubernetes API.

Field	Description	Scheme
connection	Kubernetes connection for cluster access	KubernetesConnection
container	Container name. Required if pod has multiple containers	string
fieldSelector	Select pods by fields (e.g., status.phase=Running)	string
follow	Stream logs in real-time (for long-running playbooks)	boolean
kubeconfig	Kubeconfig content or path	EnvVar
labelSelector	Select pods by labels (e.g., app=myservice,env=prod)	string
namespace	Kubernetes namespace	string
pod	Pod name (exact match)	string
previous	Fetch logs from previous (crashed) container instance	boolean
since	Relative duration (e.g., 1h, 30m). Logs newer than this	string
sinceTime	Absolute time (RFC3339). Logs after this timestamp	string
tailLines	Number of lines from the end of logs to retrieve	integer
timestamps	Prepend RFC3339 timestamp to each log line	boolean

Example: Fetch logs for a specific pod

k8s-pod-logs.yaml
apiVersion: mission-control.flanksource.com/v1
kind: Playbook
metadata:
  name: get-pod-logs
spec:
  on:
    config:
      - types:
          - Kubernetes::Pod
  actions:
    - name: fetch-logs
      logs:
        kubernetes:
          namespace: '{{.config.tags.namespace}}'
          pod: '{{.config.name}}'
          since: 1h
          tailLines: 1000
          timestamps: true

Example: Fetch logs from crashed container

k8s-crashloop-logs.yaml
apiVersion: mission-control.flanksource.com/v1
kind: Playbook
metadata:
  name: debug-crashloop
spec:
  on:
    config:
      - types:
          - Kubernetes::Pod
        filter: config.status == "CrashLoopBackOff"
  actions:
    - name: current-logs
      logs:
        kubernetes:
          namespace: '{{.config.tags.namespace}}'
          pod: '{{.config.name}}'
          tailLines: 200
    - name: previous-logs
      logs:
        kubernetes:
          namespace: '{{.config.tags.namespace}}'
          pod: '{{.config.name}}'
          previous: true
          tailLines: 200

Example: Fetch logs by label selector

k8s-label-selector-logs.yaml
apiVersion: mission-control.flanksource.com/v1
kind: Playbook
metadata:
  name: service-logs
spec:
  parameters:
    - name: service
      label: Service Name
  actions:
    - name: fetch-all-pods
      logs:
        kubernetes:
          namespace: production
          labelSelector: 'app={{.params.service}}'
          since: 30m
          tailLines: 100

Loki

Query logs from Grafana Loki using LogQL.

Field	Description	Scheme
query*	LogQL query (e.g., {app="myservice"} |= "error")	string
connection	Connection name for Loki credentials	Connection
direction	Query direction: forward (oldest first) or backward (newest first)	forward | backward
end	End time - relative or absolute. Defaults to now	string
limit	Maximum number of log entries to return	integer
password	Basic auth password	EnvVar
start	Start time - relative (e.g., 1h, 24h) or absolute (RFC3339)	string
url	Loki server URL (e.g., http://loki:3100)	string
username	Basic auth username	EnvVar

Example

loki-error-logs.yaml
apiVersion: mission-control.flanksource.com/v1
kind: Playbook
metadata:
  name: fetch-loki-errors
spec:
  actions:
    - name: get-errors
      logs:
        loki:
          url: http://loki.monitoring:3100
          query: '{namespace="production"} |= "error" | json'
          start: 1h
          limit: 100
          direction: backward
        match:
          - 'severity in ["error", "fatal"]'

OpenSearch

Query logs from OpenSearch or Elasticsearch clusters.

Field	Description	Scheme
index*	Index name or pattern (e.g., logs-*, app-logs-2024.01.*)	string
connection	Connection name for OpenSearch credentials	Connection
from	Start time for query range	string
password	Basic auth password	EnvVar
query	OpenSearch query DSL or Lucene query string	string
size	Maximum number of documents to return	integer
sort	Sort configuration (e.g., [{"@timestamp": "desc"}])	[]map[string]any
timeField	Field containing timestamp. Defaults to @timestamp	string
to	End time for query range. Defaults to now	string
url	OpenSearch cluster URL	string
username	Basic auth username	EnvVar

Example

opensearch-app-logs.yaml
apiVersion: mission-control.flanksource.com/v1
kind: Playbook
metadata:
  name: search-opensearch
spec:
  actions:
    - name: search-errors
      logs:
        opensearch:
          connection: connection://opensearch/logs
          index: 'app-logs-*'
          query: 'level:error AND service:{{.config.name}}'
          from: 24h
          size: 500
          sort:
            - '@timestamp': desc
        mapping:
          timestamp: ['@timestamp']
          message: ['message', 'msg']
          severity: ['level', 'log_level']

Output

The action returns a structured result containing the retrieved logs.

Field	Description	Scheme
count	Total number of log entries returned	integer
logs	Array of log entries with normalized fields (timestamp, message, severity, etc.)	[]LogEntry

Using Results in Subsequent Actions

actions:
  - name: fetch-logs
    logs:
      kubernetes:
        namespace: production
        labelSelector: app=api
        since: 1h
  - name: analyze
    if: actions.fetch-logs.result.count > 0
    ai:
      prompt: |
        Analyze these {{.actions.fetch-logs.result.count}} log entries:
        {{.actions.fetch-logs.result.logs | toJSON}}

Templating

CEL Expressions

The following variables can be used within the CEL expressions of filter, if, delays and parameters.default:

Field	Description	Schema
config	Config passed to the playbook	ConfigItem
check	Canary Check passed to the playbook	Check
playbook	Playbook passed to the playbook	Playbook
run	Current run	Run
params	User provided parameters to the playbook	map[string]any
request	Webhook request	Webhook Request
env	Environment variables defined on the playbook	map[string]any
user.name	Name of the user who invoked the action	string
user.email	Email of the user who invoked the action	string
agent.id	ID of the agent the resource belongs to.	string
agent.name	Name of the agent the resource belongs to.	string

Conditionally Running Actions

Playbook actions can be selectively executed based on CEL expressions. These expressions must either return

• a boolean value (true indicating run the action & skip the action otherwise)
• or a special function among the ones listed below

Function	Description
always()	run no matter what; even if the playbook is cancelled/fails
failure()	run if any of the previous actions failed
skip()	skip running this action
success()	run only if all previous actions succeeded (default)
timeout()	run only if any of the previous actions timed out

delete-kubernetes-pod.yaml
---
apiVersion: mission-control.flanksource.com/v1
kind: Playbook
metadata:
  name: notify-send-with-filter
spec:
  parameters:
    - name: message
      label: The message for notification
      default: '{{.config.name}}'
  configs:
    - types: 
        - Kubernetes::Pod
  actions:
    - name: Send notification
      exec:
        script: notify-send "{{.config.name}} was created"
    - name: Bad script
      exec:
        script: deltaforce
    - name: Send all success notification
      if: success() # this filter practically skips this action as the second action above always fails
      exec:
        script: notify-send "Everything went successfully"
    - name: Send notification regardless
      if: always()
      exec:
        script: notify-send "a Pod config was created"

Defaulting Parameters

delete-kubernetes-pod.yaml
apiVersion:
  mission-control.flanksource.com/v1
  kind: Playbook
  metadata:
    name: edit
  spec:
    title: 'Edit Kustomize Resource'
    icon: flux
    parameters:
    - default: 'chore: update $(.config.type)/$(.config.name)'
      name: commit_message

Go Templating

When templating actions with Go Templates, the context variables are available as fields of the template's context object . eg .config, .user.email

Templating Actions

delete-kubernetes-pod.yaml
apiVersion: mission-control.flanksource.com/v1
kind: Playbook
metadata:
  name: scale-deployment
spec:
  description: Scale Deployment
  configs:
    - types:
        - Kubernetes::Deployment
  parameters:
    - name: replicas
      label: The new desired number of replicas.
  actions:
    - name: kubectl scale
      exec:
        script: |
          kubectl scale --replicas={{.params.replicas}} \
            --namespace={{.config.tags.namespace}} \
            deployment {{.config.name}}

Functions

Function	Description	Return
getLastAction()	Returns the result of the action that just run	Action Specific
getAction({action})	Return the result of a specific action	Action Specific

Printing out Results

Reusing Action Results

action-results.yaml
apiVersion: mission-control.flanksource.com/v1
kind: Playbook
metadata:
  name: use-previous-action-result
spec:
  description: Creates a file with the content of the config
  configs:
    - types:
        - Kubernetes::Pod
  actions:
    - name: Fetch all changes
      sql:
        query: SELECT id FROM config_changes WHERE config_id = '{{.config.id}}'
        driver: postgres
        connection: connection://postgres/local
    - name: Send notification
      if: 'last_result().count > 0'
      notification:
        title: 'Changes summary for {{.config.name}}'
        connection: connection://slack/flanksource
        message: |
          {{$rows:=index last_result "count"}}
          Found {{$rows}} changes