Plugins

Scrape Plugins are resources that modify the behavior of other scrapers. They can transform changes, add properties to configs, and even set up relationships between configs. These behaviors can also be defined on a ScrapeConfig, but using a scrape plugin makes them global, eliminating the need to define them in all scrapers.

For example, if you want to exclude all events with severity="info", a scrape plugin can help you achieve that.

plugin-change-exclusion.yaml
apiVersion: configs.flanksource.com/v1
kind: ScrapePlugin
metadata:
  name: exclude-info-level-changes
  namespace: mc
spec:
  changes:
    exclude:
      - severity == "info"

Spec

Field	Description	Scheme
changes.exclude	Ignore changes	[]CEL with Change Context
changes.mapping	Categorize changes	Mapping
properties	Custom templatable properties for the scraped config items.	[]ConfigProperty
relationship	Select all the components to link to this check	Relationships
retention	Global retention settings for config items and changes	Retention

Mapping

When you encounter a diff change, unlike an event-based change, it can sometimes appear unclear. The summary of the change may not immediately indicate its purpose. For example, the change 'status.images' might not be self-explanatory. To clarify this, you can assign types to these diff changes using mapping.

plugin-change-mapping.yaml
apiVersion: configs.flanksource.com/v1
kind: ScrapePlugin
metadata:
  name: change-mapping-rules 
  namespace: mc
spec:
  changes:
    mapping:
      - filter: >
          change_type == 'diff' && summary == "status.containerStatuses" &&
          patch != null && has(patch.status) && has(patch.status.containerStatuses) &&
          patch.status.containerStatuses.size() > 0 &&
          has(patch.status.containerStatuses[0].restartCount)
        type: PodCrashLooping
      - filter: >
          change_type == 'diff' && summary == "status.images" && config.kind == "Node"
        type: ImageUpdated

Field	Description	Scheme
action	What action to take on the change, if delete then the corresponding config item is marked as deleted	delete | ignore
filter	Selects changes to apply the mapping	CEL
summary	New summary of the change	Go Template
type	New change type	string

Retention

Scrape plugins can define global retention policies that are merged with individual scraper retention settings. This is useful for enforcing organization-wide retention policies.

apiVersion: configs.flanksource.com/v1
kind: ScrapePlugin
metadata:
  name: global-retention
spec:
  retention:
    changes:
      - name: PullSucceeded
        age: 7d
      - name: diff
        age: 30d
    types:
      - name: Kubernetes::Pod
        deletedAge: 7d
    staleItemAge: 24h

When a scraper runs, retention settings from all applicable plugins are merged with the scraper's own retention configuration. See Retention for the full schema.