Terraform: AWS ACM Certificates for Multiple Domains

My life got better when AWS introduced Certificate Manager, their service for issuing validated TLS certificates for consumption directly by other AWS services. You don’t get to download certificates issued by ACM to install on your own servers, but you can use them with your EC2 Load Balancers, CloudFront and some other services, alleviating the need to upload certificates and renew them since ACM renews them automatically.

Closing the loop on automated certificates however, was still difficult since domain validation was done through verification emails. In Nov 2017 ACM started supporting DNS validation, which is especially great if your DNS resides on Route53. Looking to drive this combination with a single workflow, I looked at Terraform and happily enough, it supports all requisite services to make this happen. Let’s take a look.

resource "aws_acm_certificate" "main" {
  domain_name = "example.net"
  subject_alternative_names = ["*.example.net"]
  validation_method = "DNS"
  tags {
    Name = "example.net"
    terraform = "true"
  }
}

data "aws_route53_record" "validation" {
  name = "example.net."
}

resource "aws_route53_record" "validation" {
  name = "${aws_acm_certificate.main.domain_validation_options[0].resource_record_name}"
  type = "${aws_acm_certificate.main.domain_validation_options[0].resource_record_type}"
  zone_id = "${data.aws_route53_zone.validation.zone_id}"
  records = ["${aws_acm_certificate.main.domain_validation_options[0].resource_record_value}"]
  ttl = 60
}

resource "aws_acm_certificate_validation" "main" {
  certificate_arn = "${aws_acm_certificate.main.arn}"
  validation_record_fqdns = ["${aws_route53_record.validation.*.fqdn}"]
}

In the basic workflow of a wildcard certificate for a single domain, Terraform first requests a certificate, then creates validation records in DNS using the zone it looked up, then goes back to ACM to request validation. Importantly, Terraform then waits for the validation to complete before continuing, a crucial point that makes it possible to immediately start using this certificate elsewhere with Terraform without racing against the validation process.

This is pretty great, but it’s not yet portable, and what if we want to exploit all 10 (yes, ten) subjectAlternativeNames that ACM offers us?

I toyed with this for some time, getting angry and then sad, but eventually elated, at Terraform’s interpolation functions, until I came up with this (excerpt from a working Terraform module):

variable "domain_names" { type = "list" }
variable "zone_id" {}

resource "aws_acm_certificate" "main" {
  domain_name = "${var.domain_names[0]}"
  subject_alternative_names = "${slice(var.domain_names, 1, length(var.domain_names))}"
  validation_method = "DNS"
  tags {
    Name = "${var.domain_names[0]}"
    terraform = "true"
  }
}

resource "aws_route53_record" "validation" {
  count = "${length(var.domain_names)}"
  name = "${lookup(aws_acm_certificate.main.domain_validation_options[count.index], "resource_record_name")}"
  type = "${lookup(aws_acm_certificate.main.domain_validation_options[count.index], "resource_record_type")}"
  zone_id = "${var.zone_id}"
  records = ["${lookup(aws_acm_certificate.main.domain_validation_options[count.index], "resource_record_value")}"]
  ttl = 60
}

resource "aws_acm_certificate_validation" "main" {
  certificate_arn = "${aws_acm_certificate.main.arn}"
  validation_record_fqdns = ["${aws_route53_record.validation.*.fqdn}"]
}

output "arn" {
  value = "${aws_acm_certificate.main.arn}"
}

Use this as a module like this:

module "acm_ops" {
  source = "modules/aws_acm_certificate"
  domain_names = ["ops.acme.net", "*.ops.acme.net"]
  zone_id = "${aws_route53_zone.external.id}"
}

module "acm_marketing" {
  source = "modules/aws_acm_certificate"
  domain_names = ["acme.com", "*.acme.com"]
  zone_id = "${aws_route53_zone.acme.id}"
}

The module accepts a list of domain names and a Route53 zone ID, and will generate a unified validated certificate, returning the ARN of the certificate which you can then use with your ELB or CloudFront resources. Peeking inside, this makes use of lookup() and splatting to parse the validation options and create all the necessary DNS records.

The full source code for this module is available on GitHub.

tom

April 12, 2018

Uncategorized

acm, aws, certificates, security, terraform

Terraform: Cross Account S3 Bucket Access Control

Whilst auditing a set of organizational AWS accounts, I wanted to consolidate operational S3 buckets into a single account and grant access as required. It might not be immediately obvious the first time you do this, so this post is a bit of a primer on cross-account S3 access control, and implementing such with Terraform.

Connecting a remote IAM principle to an S3 bucket involves two distinct steps. First you create a trust relationship with the remote AWS account by specifying the account ID in the S3 bucket policy. Lastly, the remote AWS account may then delegate access to its IAM users (or roles) by specifying the bucket name in a policy. Because the S3 namespace is global, policies in the remote account can resolve the bucket by name.

The S3 bucket policy might look something like this.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "1",
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::123456789012:root"
                ]
            },
            "Action": [
                "s3:ListBucket",
                "s3:GetObjectVersion",
                "s3:GetObject",
                "s3:GetBucketVersioning",
                "s3:GetBucketLocation"
            ],
            "Resource": [
                "arn:aws:s3:::the-private-bucket/*",
                "arn:aws:s3:::the-private-bucket"
            ]
        }
    ]
}

In this example, read-only access to the bucket the-private-bucket is delegated to the AWS account 123456789012. The specific principal referenced is the root user of that account, but this is effective for any IAM user/role on that account having access specifically granted via an IAM policy.

Such an IAM policy might look something like this.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "1",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetObjectVersion",
                "s3:GetObject",
                "s3:GetBucketVersioning",
                "s3:GetBucketLocation"
            ],
            "Resource": [
                "arn:aws:s3:::the-private-bucket/*",
                "arn:aws:s3:::the-private-bucket"
            ]
        }
    ]
}

Remember this policy should be defined on the remote (delegated) AWS account, and thus attached to any IAM principal in that account where access is being granted.

After proving your setup by testing out your variation of the above policies, you can model this with Terraform. By defining both AWS accounts as Terraform providers, you can have Terraform manage this for you end-to-end. For completeness, my example below includes both AWS providers for the host and demo accounts, the creation of the S3 bucket, an IAM user and role, and definition & attachment of both policies.

provider "aws" {
  region = "us-east-1"
}

provider "aws" {
  alias = "demo"
  region = "us-east-1"
  profile = "demo"
}

data "aws_caller_identity" "demo" {
  provider = "aws.demo"
}

data "aws_iam_policy_document" "s3_private_bucket" {
  statement {
    sid = "1"

    actions = [
      "s3:GetBucketLocation",
      "s3:GetBucketVersioning",
      "s3:GetObject",
      "s3:GetObjectVersion",
      "s3:ListBucket",
    ]

    effect = "Allow"

    resources = [
      "arn:aws:s3:::acme-private-bucket",
      "arn:aws:s3:::acme-private-bucket",
    ]

    principals {
      type = "AWS"
      identifiers = [
        "arn:aws:iam::${data.aws_caller_identity.demo.account_id}:root",
      ]
    }
  }
}

resource "aws_s3_bucket" "private_bucket" {
  bucket = "acme-private-bucket"
  acl = "private"
  policy = "${data.aws_iam_policy_document.s3_private_bucket.json}"

  tags {
    Name = "acme-private-bucket"
    terraform = "true"
  }
}

data "aws_iam_policy_document" "private_bucket" {
  statement {
    sid = "1"
    actions = [
      "s3:GetBucketLocation",
      "s3:GetBucketVersioning",
      "s3:GetObject",
      "s3:GetObjectVersion",
      "s3:ListBucket",
    ]
    effect = "Allow"
    resources = [
      "arn:aws:s3:::acme-ansible-files",
      "arn:aws:s3:::acme-ansible-files/*",
    ]
  }
}

resource "aws_iam_policy" "private_bucket_demo" {
  provider = "aws.demo"
  name = "acme-private-bucket"
  policy = "${data.aws_iam_policy_document.private_bucket.json}"
}

resource "aws_iam_role" "demo" {
  provider = "aws.demo"
  name = "demo"
  assume_role_policy = <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": "sts:AssumeRole",
            "Principal": {
               "Service": "ec2.amazonaws.com"
            },
            "Effect": "Allow",
            "Sid": ""
        }
    ]
}
EOF
}

resource "aws_iam_instance_profile" "demo" {
  provider = "aws.demo"
  name = "demo"
  role = "${aws_iam_role.demo.name}"
}

resource "aws_iam_role_policy_attachment" "demo_private_bucket" {
  provider = "aws.demo"
  role = "${aws_iam_role.demo.name}"
  policy_arn = "${aws_iam_policy.private_bucket_demo.arn}"
}

resource "aws_iam_user" "demo" {
  name = "demo"
}

resource "aws_iam_user_policy_attachment" "demo_private_bucket" {
  user = "${aws_iam_user.demo.name}"
  policy_arn = "${aws_iam_policy.private_bucket_demo.arn}"
}

tom

February 24, 2018

Uncategorized

aws, s3, security, terraform