Plugins

Scrape Plugins are resources that modify the behavior of other scrapers. They can transform changes, add properties to configs, and even set up relationships between configs. These behaviors can also be defined on a ScrapeConfig, but using a scrape plugin makes them global, eliminating the need to define them in all scrapers.

For example, if you want to exclude all events with severity="info", a scrape plugin can help you achieve that.

plugin-change-exclusion.yaml
apiVersion: configs.flanksource.com/v1
kind: ScrapePlugin
metadata:
  name: exclude-info-level-changes
  namespace: mc
spec:
  changes:
    exclude:
      - severity == "info"

Spec

Field	Description	Scheme
aliases	Add alternate identifiers used to look up config items	[]LocationOrAlias
changes.exclude	Ignore changes	[]CEL with Change Context
changes.generate	Create changes from matching config items, without replacing those items	Generate
changes.mapping	Categorize changes	Mapping
configs.mapping	Override fields on scraped config items	ConfigMapping
locations	Add location metadata for grouping	[]LocationOrAlias
properties	Custom templatable properties for the scraped config items.	[]ConfigProperty
relationship	Select all the components to link to this check	Relationships
retention	Global retention settings for config items and changes	Retention

Mapping

When you encounter a diff change, unlike an event-based change, it can sometimes appear unclear. The summary of the change may not immediately indicate its purpose. For example, the change 'status.images' might not be self-explanatory. To clarify this, you can assign types to these diff changes using mapping.

plugin-change-mapping.yaml
apiVersion: configs.flanksource.com/v1
kind: ScrapePlugin
metadata:
  name: change-mapping-rules 
  namespace: mc
spec:
  changes:
    mapping:
      - filter: >
          change_type == 'diff' && summary == "status.containerStatuses" &&
          patch != null && has(patch.status) && has(patch.status.containerStatuses) &&
          patch.status.containerStatuses.size() > 0 &&
          has(patch.status.containerStatuses[0].restartCount)
        type: PodCrashLooping
      - filter: >
          change_type == 'diff' && summary == "status.images" && config.kind == "Node"
        type: ImageUpdated

Field	Description	Scheme
action	What action to take on the change. See Change Mapping	delete | ignore | move-up | copy-up | move | copy
filter	Selects changes to apply the mapping	CEL
summary	New summary of the change	Go Template
type	New change type	string

Retention

Scrape plugins can define global retention policies that are merged with individual scraper retention settings. This is useful for enforcing organization-wide retention policies.

apiVersion: configs.flanksource.com/v1
kind: ScrapePlugin
metadata:
  name: global-retention
spec:
  retention:
    changes:
      - name: PullSucceeded
        age: 7d
      - name: diff
        age: 30d
    types:
      - name: Kubernetes::Pod
        deletedAge: 7d
    staleItemAge: 24h

When a scraper runs, retention settings from all applicable plugins are merged with the scraper's own retention configuration. See Retention for the full schema.