AWS

This config type is used to scrape information about your AWS infrastructure.

Registry

The Mission Control Registry includes an AWS Helm chart that provides a pre-configured Scraper with common defaults

aws-scraper.yaml
apiVersion: configs.flanksource.com/v1
kind: ScrapeConfig
metadata:
  name: aws-scraper
spec:
  aws:
    - region:
        - eu-west-1
      exclude:
        - Amazon EC2 Reserved Instances Optimization
        - Savings Plan
        - trusted_advisor
        - cloudtrail

Field	Description	Scheme	Required
schedule	Specify the interval to scrape in cron format. Defaults to every 60 minutes.	Cron
retention	Settings for retaining changes, analysis and scraped items	Retention
aws	Specifies the list of AWS configurations to scrape.	[]AWS

AWS

Field	Description	Scheme
accessKey	Access Key ID	EnvVar
assumeRole	Role ARN to assume	string
cloudtrail	Ingest cloudtrail events	CloudTrail
compliance	Toggle scraping of compliance metadata	boolean
connection	The connection url to use, mutually exclusive with accessKey and secretKey	Connection
costReporting	Enable cost and usage reporting	CostReporting
endpoint	Custom AWS Endpoint to use	string
exclude	AWS resources to exclude from scraping	[]string
exclusions	Exclude individual scraped items by type, name and/or tags	[]Exclusion
include	AWS resources to include for scraping	[]string
region	The AWS region	string
secretKey	Secret Access Key	EnvVar
sessionToken	Session token	EnvVar
skipTLSVerify	Skip TLS verify when connecting to AWS	boolean
labels	Labels for each config item.	map[string]string
properties	Custom templatable properties for the scraped config items.	[]ConfigProperty
tags	Tags for each config item. Max allowed: 5	[]ConfigTag
transform	Transform configs after they've been scraped	Transform

Exclusions

Where exclude drops whole resource types, exclusions drops individual items. A rule matches when all of its populated fields match, and an item is excluded when any rule matches.

type and name accept comma-separated patterns and support globs (*, prefix*, *suffix*) and negation (!pattern).

Field	Description	Scheme
type	Config type patterns to exclude e.g. AWS::EC2::Instance	string
name	Name patterns to exclude	string
tags	Tags the item must carry, values matched as patterns	map[string]string

aws-exclusions.yaml
aws:
  - exclusions:
      # Drop every scratch bucket
      - type: AWS::S3::Bucket
        name: 'scratch-*'
      # Drop anything tagged as ephemeral
      - tags:
          lifecycle: ephemeral

CloudTrail

Field	Description	Scheme
exclude	Set events to be excluded from scraping	[]string
maxAge	Set maximum age of events for scraping, Defaults to 7d	Duration

Cost Reporting

Field	Description	Scheme
s3BucketPath	Set path for S3 bucket to scrape published AWS billing reports	string
table	Specify table containing cost and usage data	string
database	Specify database containing cost and usage data	string
region	Specify region for S3 bucket	string
lookbackDays	How many days of the cost and usage report to read on each scrape. Values of 0 or less use the 45 day default	int

Trusted Advisor

The scraper integrates with AWS Trusted Advisor to collect real-time guidance across cost optimization, performance, fault tolerance, security, and service limits to help optimize your AWS infrastructure.

Rate Limitations

Trusted Advisor checks run with a minimum interval of 16 hours (due to API rate limitations). This can be overridden by setting the property scraper.aws.trusted_advisor.minInterval.

To disable Trusted Advisor altogether, you can add an exclusion rule:

exclude:
  - trusted_advisor

Supported Resources

Resource Type	AWS Type	Config Class	Description
Account	AWS::IAM::Account	Account	AWS Account information
CloudFormationStack	AWS::CloudFormation::Stack	Stack	CloudFormation stacks
DHCPOptions	AWS::EC2::DHCPOptions	DHCP	DHCP Options Sets
DNSZone	AWS::Route53::HostedZone	DNSZone	Route53 Hosted Zones
EBSVolume	AWS::EBS::Volume	DiskStorage	Elastic Block Store Volumes
EC2Instance	AWS::EC2::Instance	VirtualMachine	EC2 Instances
ECRRepository	AWS::ECR::Repository	ContainerRegistry	Elastic Container Registry Repositories
ECSCluster	AWS::ECS::Cluster	ECSCluster	ECS Clusters
ECSService	AWS::ECS::Service	ECSService	ECS Services
ECSTask	AWS::ECS::Task	ECSTask	ECS Tasks
ECSTaskDefinition	AWS::ECS::TaskDefinition	ECSTaskDefinition	ECS Task Definitions
EFSFileSystem	AWS::EFS::FileSystem	FileSystem	Elastic File System
EKSCluster	AWS::EKS::Cluster	KubernetesCluster	Elastic Kubernetes Service Clusters
ElastiCache	AWS::ElastiCache::CacheCluster	Cache	ElastiCache Clusters
FargateProfile	AWS::EKS::FargateProfile	FargateProfile	EKS Fargate Profiles
IAMInstanceProfile	AWS::IAM::InstanceProfile	Profile	IAM Instance Profiles
IAMRole	AWS::IAM::Role	Role	IAM Roles
IAMUser	AWS::IAM::User	User	IAM Users
LambdaFunction	AWS::Lambda::Function	Lambda	Lambda Functions
LoadBalancer	AWS::ElasticLoadBalancing::LoadBalancer	LoadBalancer	Classic Load Balancers
LoadBalancerV2	AWS::ElasticLoadBalancingV2::LoadBalancer	LoadBalancer	Application/Network Load Balancers
RDSInstance	AWS::RDS::DBInstance	RelationalDatabase	RDS Database Instances
RouteTable	AWS::EC2::RouteTable	Route	VPC Route Tables
S3Bucket	AWS::S3::Bucket	ObjectStorage	S3 Buckets
SecurityGroup	AWS::EC2::SecurityGroup	SecurityGroup	Security Groups
SNSTopic	AWS::SNS::Topic	Topic	Simple Notification Service Topics
SQSQueue	AWS::SQS::Queue	Queue	Simple Queue Service Queues
Subnet	AWS::EC2::Subnet	Subnet	VPC Subnets
VPC	AWS::EC2::VPC	VPC	Virtual Private Clouds