Jan 14, 2020

Keeping Secrets Out of Terraform State

There are many instances where you will want to create resources via Terraform with secrets that you just don’t want anyone to see. These could be IAM credentials, certificates, RDS DB credentials, etc. One problem with creating these resources via Terraform is that the statefile is stored in plain text. Yes, you could treat that statefile as a secret itself, but that may not be good enough depending on your security requirements. For example, you may want to be able to share that statefile with other users that are not authorized to view the secret’s values.

I will show a simple example of one way solve this problem. It may seem obvious, but the various cloud provider’s SDKs and APIs usually include a way to set these values. I’ll show here how to set up a new RDS instance and keep its master password from prying eyes.

First, I will create an RDS DB instance.

resource "aws_db_instance" "default" {
  name                 = "postgres_db"
  engine               = "postgres"
  instance_class       = "db.t2.micro"
  allocated_storage    = 20
  storage_type         = "gp2"
  username             = "foo"
  password             = "password123"
}

The password here is just a placeholder to initialize the instance. Next, I’ll use the boto3 Python library to update that password to a more secure secret.

#!/usr/env/bin python3
 
import argparse
import boto3
import secrets
import string
import time
 
parser = argparse.ArgumentParser(description='Override Terraform password')
parser.add_argument('--db_identifier', type=str, required=True, help="RDS DB instance identifier")
parser.add_argument('--secret_name', type=str, default='db_password', required=False, help="Name for SSM Parameter")
parser.add_argument('--region', type=str, default='us-east-1', required=False, help="AWS region name")
args = parser.parse_args()
 
 
def generate_password(string_length=10):
    """Generate a secure random string of letters, digits and special characters """
    password_characters = string.ascii_letters + string.digits + "!#$%^&*()"
    return ''.join(secrets.choice(password_characters) for i in range(string_length))
 
 
db_id = args.db_identifier
db_password = generate_password(16)
secret_name = args.secret_name
region = args.region
 
rds = boto3.client('rds', region_name=region)
db = rds.describe_db_instances(DBInstanceIdentifier=db_id)
while rds.describe_db_instances(DBInstanceIdentifier=db_id)["DBInstances"][0]["DBInstanceStatus"] != "available":
    time.sleep(3)
response = rds.modify_db_instance(DBInstanceIdentifier=db_id, MasterUserPassword=db_password, ApplyImmediately=True)
if response:
    print(f"Updated RDS password for {db_id}")

Using the script above, we will no longer have our “real” password stored in state. However, if we ever want to log into this DB we will need to store this somewhere else. This could be whatever tooling you use to manage secrets (Ex. Vault, CyberArk, etc.). If we want to keep this example native to AWS, we could store it in SSM Parameter Store by adding the following to our Python script:

ssm = boto3.client('ssm')
ssm.put_parameter(
    Name=secret_name,
    Description=f'DB Password for {db_id}',
    Type='SecureString',
    Value=db_password,
    Overwrite=True
)

All that’s left is getting Terraform to handle these actions through a null_resource. This could be a local_exec or remote_exec depending where you have dependencies available for the script.

resource "null_resource" "update_password" {
  provisioner "local-exec" {
    command = "python3 update_password.py --db_identifier=${aws_db_instance.default.identifier}"
  }
}

This Python script above is most useful if you need to store these secrets somewhere outside of AWS, rather than putting them in Parameter Store. A better AWS alternative might be to utilize AWS Secrets Manager to generate a password for RDS. Secrets Manager can also handle automatic rotation. The only real trade off with using Secrets Manager is that you will incur cost.

You can view my directory structure and full text of this example on GitHub here

I hope this sparks some creative ways to maintain your secrecy!

About the Author

Andy Haynssen profile.

Andy Haynssen

Principal Technologist

Andy is a creative person with a passion for software and technology! He has experience in building enterprise platforms with a focus on Infrastructure as Code, public cloud, and Continuous Integration/Continuous Delivery. He has been focusing in the DevOps space for the past 5+ years. In his free time, Andy enjoys creating music, woodworking, and spending time outdoors.

Leave a Reply

Your email address will not be published. Required fields are marked *

Related Blog Posts
An Exploration in Rust: Musings From a Java/C++ Developer
Why Rust? It’s fast (runtime performance) It’s small (binary size) It’s safe (no memory leaks) It’s modern (build system, language features, etc) When Is It Worth It? Embedded systems (where it is implied that interpreted […]
Getting Started with CSS Container Queries
For as long as I’ve been working full-time on the front-end, I’ve heard about the promise of container queries and their potential to solve the majority of our responsive web design needs. And, for as […]
Simple improvements to making decisions in teams
Software development teams need to make a lot of decisions. Functional requirements, non-functional requirements, user experience, API contracts, tech stack, architecture, database schemas, cloud providers, deployment strategy, test strategy, security, and the list goes on. […]
JavaScript Bundle Optimization – Polyfills
If you are lucky enough to only support a small subset of browsers (for example, you are targeting a controlled set of users), feel free to move along. However, if your website is open to […]