# Terraform

The Terraform scraper reads Terraform state files and creates configuration items from the resources defined in the state. This enables you to track your infrastructure as code resources and monitor changes to your Terraform-managed infrastructure.

## Use Cases[​](#use-cases "Direct link to Use Cases")

* **Infrastructure Tracking**: Monitor all resources managed by Terraform
* **State Drift Detection**: Track changes between Terraform state and actual infrastructure
* **Multi-Environment Management**: Track resources across different environments and stages
* **Resource Dependency Mapping**: Understand relationships between Terraform-managed resources
* **Compliance Monitoring**: Ensure infrastructure remains compliant with defined configurations

```
terraform-scraper.yaml
---

apiVersion: configs.flanksource.com/v1

kind: ScrapeConfig

metadata:

  name: terraform

spec:

  schedule: '@every 5m'

  terraform:

    - name: '{{ filepath.Base .path}}'

      state:

        s3:

          bucket: terraform

          connection: connection://aws

          objectPath: 'states/**/*.tfstate'
```

| Field       | Description                                                                  | Scheme                                                     | Required |
| ----------- | ---------------------------------------------------------------------------- | ---------------------------------------------------------- | -------- |
| `schedule`  | Specify the interval to scrape in cron format. Defaults to every 60 minutes. | [Cron](/docs/reference/types.md#cron)                      |          |
| `retention` | Settings for retaining changes, analysis and scraped items                   | [`Retention`](/docs/guide/config-db/concepts/retention.md) |          |
| `terraform` | Specifies the list of Terraform configurations to scrape.                    | [`[]Terraform`](#terraform)                                | `true`   |

### Terraform[​](#terraform "Direct link to Terraform")

| Field       | Description                                                                  | Scheme                                                   |
| ----------- | ---------------------------------------------------------------------------- | -------------------------------------------------------- |
| `name*`     | Template for naming the configuration items created from Terraform resources | [GoTemplate](/docs/reference/types.md#gotemplate)        |
| `state*`    | Configuration for accessing Terraform state files                            | [TerraformStateSource](#terraformstatesource)            |
| `transform` | Transformations to apply to scraped items                                    | [Transform](/docs/guide/config-db/concepts/transform.md) |

### TerraformStateSource[​](#terraformstatesource "Direct link to TerraformStateSource")

| Field   | Description                                                          | Scheme                          |
| ------- | -------------------------------------------------------------------- | ------------------------------- |
| `gcs`   | Google Cloud Storage backend configuration for accessing state files | [GCSConnection](#gcsconnection) |
| `local` | Local file path to Terraform state file                              | `string`                        |
| `s3`    | S3 backend configuration for accessing state files stored in AWS S3  | [S3Connection](#s3connection)   |

### S3Connection[​](#s3connection "Direct link to S3Connection")

| Field         | Description                                                                      | Scheme                                    |
| ------------- | -------------------------------------------------------------------------------- | ----------------------------------------- |
| `bucket*`     | S3 bucket name containing the state files                                        | `string`                                  |
| `objectPath*` | Path pattern to state files (supports glob patterns like states/\*\*/\*.tfstate) | `string`                                  |
| `accessKey`   | AWS access key for authentication                                                | [EnvVar](/docs/reference/types.md#envvar) |
| `connection`  | Connection reference for AWS credentials                                         | `string`                                  |
| `region`      | AWS region where the bucket is located                                           | `string`                                  |
| `secretKey`   | AWS secret key for authentication                                                | [EnvVar](/docs/reference/types.md#envvar) |

### GCSConnection[​](#gcsconnection "Direct link to GCSConnection")

| Field         | Description                                | Scheme                                    |
| ------------- | ------------------------------------------ | ----------------------------------------- |
| `bucket*`     | GCS bucket name containing the state files | `string`                                  |
| `objectPath*` | Path pattern to state files in the bucket  | `string`                                  |
| `credentials` | GCP service account credentials JSON       | [EnvVar](/docs/reference/types.md#envvar) |

## Configuration Examples[​](#configuration-examples "Direct link to Configuration Examples")

### S3 Backend[​](#s3-backend "Direct link to S3 Backend")

```
apiVersion: configs.flanksource.com/v1

kind: ScrapeConfig

metadata:

  name: terraform-s3-states

spec:

  schedule: '@every 5m'

  terraform:

    - name: '{{ filepath.Base .path }}'

      state:

        s3:

          bucket: terraform-state-bucket

          connection: connection://aws

          objectPath: 'environments/**/*.tfstate'
```

### Multiple Environment Tracking[​](#multiple-environment-tracking "Direct link to Multiple Environment Tracking")

```
apiVersion: configs.flanksource.com/v1

kind: ScrapeConfig

metadata:

  name: terraform-multi-env

spec:

  terraform:

    - name: 'prod-{{ filepath.Base .path }}'

      state:

        s3:

          bucket: terraform-prod-states

          connection: connection://aws-prod

          objectPath: 'prod/**/*.tfstate'

    - name: 'staging-{{ filepath.Base .path }}'

      state:

        s3:

          bucket: terraform-staging-states

          connection: connection://aws-staging

          objectPath: 'staging/**/*.tfstate'

    - name: 'dev-{{ filepath.Base .path }}'

      state:

        s3:

          bucket: terraform-dev-states

          connection: connection://aws-dev

          objectPath: 'dev/**/*.tfstate'
```

### GCS Backend[​](#gcs-backend "Direct link to GCS Backend")

```
apiVersion: configs.flanksource.com/v1

kind: ScrapeConfig

metadata:

  name: terraform-gcs-states

spec:

  terraform:

    - name: 'gcp-{{ filepath.Base .path }}'

      state:

        gcs:

          bucket: terraform-state-gcs

          objectPath: 'projects/**/*.tfstate'

          credentials:

            valueFrom:

              secretKeyRef:

                name: gcp-credentials

                key: service-account.json
```

### Local State Files[​](#local-state-files "Direct link to Local State Files")

```
apiVersion: configs.flanksource.com/v1

kind: ScrapeConfig

metadata:

  name: terraform-local-state

spec:

  terraform:

    - name: 'local-terraform'

      state:

        local: '/path/to/terraform.tfstate'
```

## Masking Sensitive Data[​](#masking-sensitive-data "Direct link to Masking Sensitive Data")

Terraform state files store all resource attributes in plain text JSON, including sensitive values like passwords, API keys, and connection strings. For example, an AWS RDS instance in the state file contains:

```
{

  "mode": "managed",

  "type": "aws_db_instance",

  "name": "example",

  "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",

  "instances": [

    {

      "attributes": {

        "address": "mydb.123456789012.us-east-1.rds.amazonaws.com",

        "engine": "mysql",

        "engine_version": "5.7",

        "master_username": "admin",

        "master_password": "MyPa$$w0rd123!",  // Stored in plain text!

        "endpoint": "mydb.123456789012.us-east-1.rds.amazonaws.com:3306"

      }

    }

  ]

}
```

### Default Masking Behavior[​](#default-masking-behavior "Direct link to Default Masking Behavior")

info

The Terraform scraper **automatically masks** common sensitive patterns by default. You don't need to configure anything for basic secret protection.

The following patterns are automatically masked:

| Pattern            | Fields Masked                                                | Example                                      |
| ------------------ | ------------------------------------------------------------ | -------------------------------------------- |
| Passwords          | `*password*`, `*passwd*`, `*pwd*`                            | `master_password`, `db_password`, `user_pwd` |
| Keys               | `*private_key*`, `*secret_key*`, `*api_key*`, `*access_key*` | `aws_secret_access_key`, `private_key_pem`   |
| Tokens             | `*token*`, `*auth*`                                          | `github_token`, `auth_token`, `bearer_token` |
| Secrets            | `*secret*`                                                   | `client_secret`, `webhook_secret`            |
| Connection Strings | Fields containing connection URIs                            | `connection_string`, `database_url`          |

### Custom Masking Examples[​](#custom-masking-examples "Direct link to Custom Masking Examples")

For additional masking beyond the defaults, you can add custom rules. Note that Mission Control uses specific resource type formats (e.g., `AWS::RDS::DBInstance`, `AWS::IAM::User`):

```
apiVersion: configs.flanksource.com/v1

kind: ScrapeConfig

metadata:

  name: terraform-security-masking

spec:

  terraform:

    - name: '{{ .path | filepath.Base }}'

      state:

        s3:

          bucket: terraform-states

          connection: connection://aws

          objectPath: '**/*.tfstate'

      transform:

        mask:

          # Mask RDS and Aurora passwords with hash for change detection

          - selector: |

              config.type == 'AWS::RDS::DBInstance' ||

              config.type == 'AWS::RDS::DBCluster' ||

              config.type == 'AWS::RDS::DBProxy'

            jsonpath: $.config.master_password

            value: md5sum



          # Mask DocumentDB passwords

          - selector: config.type == 'AWS::DocDB::DBCluster'

            jsonpath: $.config.master_password

            value: md5sum



          # Mask ElastiCache auth tokens

          - selector: config.type == 'AWS::ElastiCache::CacheCluster'

            jsonpath: $.config.auth_token

            value: '***REDACTED***'



          # Mask IAM user access keys

          - selector: config.type == 'AWS::IAM::User'

            jsonpath: $.config.access_key_id

            value: '***ACCESS_KEY***'



          # Mask Lambda environment variables

          - selector: config.type == 'AWS::Lambda::Function'

            jsonpath: $.config.environment.variables[?(@.key =~ /.*secret.*|.*password.*|.*key.*/i)]

            value: md5sum



          # Mask custom application secrets in EC2 user data

          - selector: config.type == 'AWS::EC2::Instance'

            jsonpath: $.config.user_data

            value: '***USER_DATA_REDACTED***'
```

### Excluding Sensitive Resources[​](#excluding-sensitive-resources "Direct link to Excluding Sensitive Resources")

For highly sensitive resources that shouldn't be tracked at all:

```
apiVersion: configs.flanksource.com/v1

kind: ScrapeConfig

metadata:

  name: terraform-exclude-secrets

spec:

  terraform:

    - name: '{{ .path | filepath.Base }}'

      state:

        s3:

          bucket: terraform-states

          connection: connection://aws

          objectPath: '**/*.tfstate'

      transform:

        exclude:

          # Exclude all AWS Secrets Manager resources

          - types:

              - 'AWS::SecretsManager::Secret'

              - 'AWS::SecretsManager::SecretVersion'

            jsonpath: '$'



          # Exclude IAM access keys

          - types:

              - 'AWS::IAM::AccessKey'

            jsonpath: '$'



          # Exclude specific sensitive fields from all resources

          - jsonpath: '$..master_password'

          - jsonpath: '$..auth_token'

          - jsonpath: '$..private_key_pem'

          - jsonpath: '$..client_secret'
```

## Best Practices[​](#best-practices "Direct link to Best Practices")

* **Leverage Default Masking**: The scraper automatically masks common sensitive patterns - verify these meet your needs before adding custom rules
* **State File Protection**: Always encrypt state files at rest and in transit, use remote backends with proper access controls
* **Use AWS Secrets Manager**: For RDS instances, use `manage_master_user_password = true` to avoid storing passwords in state
* **Regular Rotation**: Rotate credentials regularly and update masking rules accordingly
* **Audit Access**: Monitor and audit who has access to state files and scraped configurations
* **Test Masking Rules**: Regularly verify that sensitive data is properly masked in scraped configurations