AWS
This config type is used to scrape information about your AWS infrastructure.
Registry
The registry has an AWS Helm chart that provides a pre-configured Scraper with some common defaults
aws-scraper.yamlapiVersion: configs.flanksource.com/v1
kind: ScrapeConfig
metadata:
name: aws-scraper
spec:
aws:
- region:
- eu-west-2
- us-east-1
- af-south-1
- ap-south-1
- eu-central-1
properties:
- name: AWS Link
filter: 'config_type == AWS::IAM::Role'
icon: aws-iam
links:
- text: AWS Link
url: https://us-east-1.console.aws.amazon.com/iamv2/home#/roles/details/{{.name}}?section=permissions
compliance: true
patch_states: false
trusted_advisor_check: false
patch_details: false
costReporting:
s3BucketPath: s3://flanksource-cost-reports/query-results
database: athenacurcfn_flanksource_report
table: flanksource_report
region: af-south-1
inventory: true
exclude:
- Amazon EC2 Reserved Instances Optimization
- Savings Plan
# - trusted_advisor
# - cloudtrail
# include:
# - vpc
# # - subnet
# - vpc
# - SecurityGroup
transform:
relationship:
# EKS Cluster to Kubernetes Cluster & Kubernetes Node
- filter: config_type == 'AWS::EKS::Cluster'
expr: |
[
{"type": "Kubernetes::Cluster","tags": {"account": tags['account'],"cluster": labels["alpha.eksctl.io/cluster-name"]}},
{"type": "Kubernetes::Node","tags": {"account": tags['account'],"cluster": labels["alpha.eksctl.io/cluster-name"]}}
].toJSON()
# EC2 Instance to kubernetes node
- filter: config_type == 'AWS::EC2:Instance'
expr: |
[{"type": "Kubernetes::Node", "labels": {"alpha.eksctl.io/instance-id": config["instance_id"]}}].toJSON()
# IAM Role to Kubernetes Node
- filter: config_type == 'AWS::IAM::Role'
expr: |
[{"type": "Kubernetes::Node", "labels": {"aws/iam-role": config["Arn"]}}].toJSON()
# AvailabilityZone to Zone ID & Kubernetes Node
- filter: config_type == 'AWS::AvailabilityZone'
expr: |
[
{"type": "Kubernetes::Node", "tags": {"account": labels['account'], "topology.kubernetes.io/zone": name}}
].toJSON()
# Region to ZoneID
- filter: config_type == 'AWS::Region'
expr: |
[{"type": "AWS::AvailabilityZoneID", "tags": {"region": name}}].toJSON()
exclude:
- jsonpath: $.tags
- jsonpath: $.privateDnsNameOptionsOnLaunch
# - jsonpath: availableIpAddressCount
- jsonpath: outpostArn
- jsonpath: mapCustomerOwnedIpOnLaunch
- jsonpath: subnetArn
# - jsonpath: usageOperationUpdateTime
# - jsonpath: $..privateIPAddresses
Field | Description | Scheme | Required |
---|---|---|---|
schedule | Specify the interval to scrape in cron format. Defaults to every 60 minutes. | Cron | |
retention | Settings for retaining changes, analysis and scraped items | Retention | |
aws | Specifies the list of AWS configurations to scrape. | []AWS |
AWS
Field | Description | Scheme | Required |
---|---|---|---|
cloudtrail | Ingest cloudtrail events | CloudTrail | |
compliance | Toggle scraping of compliance metadata | bool | |
cost_reporting | Enable cost and usage reporting | CostReporting | |
exclude | AWS resources to exclude from scraping | []string | |
include | AWS resources to include for scraping | []string | |
properties | Custom templatable properties for the scraped config items. | []ConfigProperty | |
transform | Field to transform result | Transform | |
tags | set custom tags on the scraped config items | map[string]string |
CloudTrail
Field | Description | Scheme |
---|---|---|
exclude | Set events to be excluded from scraping | []string |
max_age | Set maximum age of events for scraping, Defaults to 7d | Duration |
Cost Reporting
Field | Description | Scheme |
---|---|---|
s3_bucket_path | Set path for S3 bucket to scrape published AWS billing reports | string |
table | Specify table containing cost and usage data | string |
database | Specify database containing cost and usage data | string |
region | Specify region for S3 bucket | string |
Supported Resources
Resource Type | AWS Type | Config Class | Description |
---|---|---|---|
Account | AWS::IAM::Account | Account | AWS Account information |
CloudFormationStack | AWS::CloudFormation::Stack | Stack | CloudFormation stacks |
DHCPOptions | AWS::EC2::DHCPOptions | DHCP | DHCP Options Sets |
DNSZone | AWS::Route53::HostedZone | DNSZone | Route53 Hosted Zones |
EBSVolume | AWS::EBS::Volume | DiskStorage | Elastic Block Store Volumes |
EC2Instance | AWS::EC2::Instance | VirtualMachine | EC2 Instances |
ECRRepository | AWS::ECR::Repository | ContainerRegistry | Elastic Container Registry Repositories |
ECSCluster | AWS::ECS::Cluster | ECSCluster | ECS Clusters |
ECSService | AWS::ECS::Service | ECSService | ECS Services |
ECSTask | AWS::ECS::Task | ECSTask | ECS Tasks |
ECSTaskDefinition | AWS::ECS::TaskDefinition | ECSTaskDefinition | ECS Task Definitions |
EFSFileSystem | AWS::EFS::FileSystem | FileSystem | Elastic File System |
EKSCluster | AWS::EKS::Cluster | KubernetesCluster | Elastic Kubernetes Service Clusters |
ElastiCache | AWS::ElastiCache::CacheCluster | Cache | ElastiCache Clusters |
FargateProfile | AWS::EKS::FargateProfile | FargateProfile | EKS Fargate Profiles |
IAMInstanceProfile | AWS::IAM::InstanceProfile | Profile | IAM Instance Profiles |
IAMRole | AWS::IAM::Role | Role | IAM Roles |
IAMUser | AWS::IAM::User | User | IAM Users |
LambdaFunction | AWS::Lambda::Function | Lambda | Lambda Functions |
LoadBalancer | AWS::ElasticLoadBalancing::LoadBalancer | LoadBalancer | Classic Load Balancers |
LoadBalancerV2 | AWS::ElasticLoadBalancingV2::LoadBalancer | LoadBalancer | Application/Network Load Balancers |
RDSInstance | AWS::RDS::DBInstance | RelationalDatabase | RDS Database Instances |
RouteTable | AWS::EC2::RouteTable | Route | VPC Route Tables |
S3Bucket | AWS::S3::Bucket | ObjectStorage | S3 Buckets |
SecurityGroup | AWS::EC2::SecurityGroup | SecurityGroup | Security Groups |
SNSTopic | AWS::SNS::Topic | Topic | Simple Notification Service Topics |
SQSQueue | AWS::SQS::Queue | Queue | Simple Queue Service Queues |
Subnet | AWS::EC2::Subnet | Subnet | VPC Subnets |
VPC | AWS::EC2::VPC | VPC | Virtual Private Clouds |