Skip to main content

AWS

This config type is used to scrape information about your AWS infrastructure.

Registry

The registry has an AWS Helm chart that provides a pre-configured Scraper with some common defaults

aws-scraper.yaml
apiVersion: configs.flanksource.com/v1
kind: ScrapeConfig
metadata:
name: aws-scraper
spec:
aws:
- region:
- eu-west-2
- us-east-1
- af-south-1
- ap-south-1
- eu-central-1
properties:
- name: AWS Link
filter: 'config_type == AWS::IAM::Role'
icon: aws-iam
links:
- text: AWS Link
url: https://us-east-1.console.aws.amazon.com/iamv2/home#/roles/details/{{.name}}?section=permissions
compliance: true
patch_states: false
trusted_advisor_check: false
patch_details: false
costReporting:
s3BucketPath: s3://flanksource-cost-reports/query-results
database: athenacurcfn_flanksource_report
table: flanksource_report
region: af-south-1
inventory: true
exclude:
- Amazon EC2 Reserved Instances Optimization
- Savings Plan
# - trusted_advisor
# - cloudtrail
# include:
# - vpc
# # - subnet
# - vpc
# - SecurityGroup
transform:
relationship:
# EKS Cluster to Kubernetes Cluster & Kubernetes Node
- filter: config_type == 'AWS::EKS::Cluster'
expr: |
[
{"type": "Kubernetes::Cluster","tags": {"account": tags['account'],"cluster": labels["alpha.eksctl.io/cluster-name"]}},
{"type": "Kubernetes::Node","tags": {"account": tags['account'],"cluster": labels["alpha.eksctl.io/cluster-name"]}}
].toJSON()
# EC2 Instance to kubernetes node
- filter: config_type == 'AWS::EC2:Instance'
expr: |
[{"type": "Kubernetes::Node", "labels": {"alpha.eksctl.io/instance-id": config["instance_id"]}}].toJSON()
# IAM Role to Kubernetes Node
- filter: config_type == 'AWS::IAM::Role'
expr: |
[{"type": "Kubernetes::Node", "labels": {"aws/iam-role": config["Arn"]}}].toJSON()
# AvailabilityZone to Zone ID & Kubernetes Node
- filter: config_type == 'AWS::AvailabilityZone'
expr: |
[
{"type": "Kubernetes::Node", "tags": {"account": labels['account'], "topology.kubernetes.io/zone": name}}
].toJSON()
# Region to ZoneID
- filter: config_type == 'AWS::Region'
expr: |
[{"type": "AWS::AvailabilityZoneID", "tags": {"region": name}}].toJSON()
exclude:
- jsonpath: $.tags
- jsonpath: $.privateDnsNameOptionsOnLaunch
# - jsonpath: availableIpAddressCount
- jsonpath: outpostArn
- jsonpath: mapCustomerOwnedIpOnLaunch
- jsonpath: subnetArn
# - jsonpath: usageOperationUpdateTime
# - jsonpath: $..privateIPAddresses
FieldDescriptionSchemeRequired
scheduleSpecify the interval to scrape in cron format. Defaults to every 60 minutes.Cron
retentionSettings for retaining changes, analysis and scraped itemsRetention
awsSpecifies the list of AWS configurations to scrape.[]AWS

AWS

FieldDescriptionSchemeRequired
cloudtrailIngest cloudtrail eventsCloudTrail
complianceToggle scraping of compliance metadatabool
cost_reportingEnable cost and usage reportingCostReporting
excludeAWS resources to exclude from scraping[]string
includeAWS resources to include for scraping[]string
propertiesCustom templatable properties for the scraped config items.[]ConfigProperty
transformField to transform resultTransform
tagsset custom tags on the scraped config itemsmap[string]string

CloudTrail

FieldDescriptionScheme
excludeSet events to be excluded from scraping[]string
max_ageSet maximum age of events for scraping, Defaults to 7dDuration

Cost Reporting

FieldDescriptionScheme
s3_bucket_pathSet path for S3 bucket to scrape published AWS billing reportsstring
tableSpecify table containing cost and usage datastring
databaseSpecify database containing cost and usage datastring
regionSpecify region for S3 bucketstring

Supported Resources

Resource TypeAWS TypeConfig ClassDescription
AccountAWS::IAM::AccountAccountAWS Account information
CloudFormationStackAWS::CloudFormation::StackStackCloudFormation stacks
DHCPOptionsAWS::EC2::DHCPOptionsDHCPDHCP Options Sets
DNSZoneAWS::Route53::HostedZoneDNSZoneRoute53 Hosted Zones
EBSVolumeAWS::EBS::VolumeDiskStorageElastic Block Store Volumes
EC2InstanceAWS::EC2::InstanceVirtualMachineEC2 Instances
ECRRepositoryAWS::ECR::RepositoryContainerRegistryElastic Container Registry Repositories
ECSClusterAWS::ECS::ClusterECSClusterECS Clusters
ECSServiceAWS::ECS::ServiceECSServiceECS Services
ECSTaskAWS::ECS::TaskECSTaskECS Tasks
ECSTaskDefinitionAWS::ECS::TaskDefinitionECSTaskDefinitionECS Task Definitions
EFSFileSystemAWS::EFS::FileSystemFileSystemElastic File System
EKSClusterAWS::EKS::ClusterKubernetesClusterElastic Kubernetes Service Clusters
ElastiCacheAWS::ElastiCache::CacheClusterCacheElastiCache Clusters
FargateProfileAWS::EKS::FargateProfileFargateProfileEKS Fargate Profiles
IAMInstanceProfileAWS::IAM::InstanceProfileProfileIAM Instance Profiles
IAMRoleAWS::IAM::RoleRoleIAM Roles
IAMUserAWS::IAM::UserUserIAM Users
LambdaFunctionAWS::Lambda::FunctionLambdaLambda Functions
LoadBalancerAWS::ElasticLoadBalancing::LoadBalancerLoadBalancerClassic Load Balancers
LoadBalancerV2AWS::ElasticLoadBalancingV2::LoadBalancerLoadBalancerApplication/Network Load Balancers
RDSInstanceAWS::RDS::DBInstanceRelationalDatabaseRDS Database Instances
RouteTableAWS::EC2::RouteTableRouteVPC Route Tables
S3BucketAWS::S3::BucketObjectStorageS3 Buckets
SecurityGroupAWS::EC2::SecurityGroupSecurityGroupSecurity Groups
SNSTopicAWS::SNS::TopicTopicSimple Notification Service Topics
SQSQueueAWS::SQS::QueueQueueSimple Queue Service Queues
SubnetAWS::EC2::SubnetSubnetVPC Subnets
VPCAWS::EC2::VPCVPCVirtual Private Clouds