Jul 1, 2021

Amazon SES Postfix Relay Docker Image

A common pattern for sending emails is to use a local Mail Transfer Agent (MTA) like Sendmail and/or Postfix on each server, then applications have applications use their respective SMTP libraries to send mail. In our case, we want to use AWS SES as a delivery agent. This helps us collect statistics, manage delivery and bounce notifications, and manage our reputation score. For this use case, Amazon provides a configuration for Postfix so that it can act as a relay to their Simple Email Service (SES).

For new development, you might want to use the SES API directly, or use a service like Sendgrid, or abstract everything behind an additional message queue. For legacy applications, SMTP is (or was?) king. For these, we followed the instructions to put the Postfix Relay on our EC2s, bundled the config into our Ansible playbooks, and ran a Postfix service on every EC2 host. This has served us well for years.

Fast forward to today, where we are feverishly modernizing infrastructure to simplify deployments, improve uptime, and reduce costs. We’ve packaged our legacy applications into containers, but we still need that local SMTP service. We’d like it to be in a container like everything else, and we don’t want to make changes to our legacy applications.

To help, we’ve created a Docker image, tmclnk/ses-postfix-relay. To use it, we’ll need to

  1. Create IAM Policy to access the Parameter Store
  2. Verify Sending Addresses or Domains
  3. Generate SMTP Credentials
  4. Test the Container Locally
  5. Deploy the container

IAM Policy

Before we get started, you’ll need:

  1. An IAM User with credentials in ~/.aws/credentials. See here for help.
  2. A policy that allows us to read/write Parameter store permissions.
{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Sid":"VisualEditor0",
         "Effect":"Allow",
         "Action":[
            "ssm:PutParameter",
            "ssm:DeleteParameter",
            "ssm:GetParameters",
            "ssm:GetParameter",
            "ssm:DeleteParameters"
         ],
         "Resource":"arn:aws:ssm:*:*:parameter/ses-relay/*"
      }
   ]
}

If you’re running with superuser permissions, you can likely skip this step for now and tighten the permissions on your environment’s IAM Role(s).

Verify Sending Addresses or Domains

To send emails from SES, you’ll need to follow the instructions here to verify identities. You can use the mailbox simulator addresses if you aren’t already using SES, or if you’re just experimenting.

Generate STMP Credentials

You will need to set up SMTP credentials via the AWS console at https://console.aws.amazon.com/ses, then switch to the SMTP Settings tab and click Create My SMTP Credentials.

When complete, you’ll have credentials like this.

You’ll need to capture the SMTP Username and SMTP Password before closing this window. Note that this has also created an IAM user named “ses-smtp-user.20210609-103815”. If you need to revoke access, the easiest thing to do is delete the IAM user.

We need a secure place to store those credentials, so we use the AWS Parameter Store.

aws ssm put-parameter --name /ses-relay/smtpusername \
  --value AKIA6MIS7W3CPR2N6SVO \
  --type String
aws ssm put-parameter --name /ses-relay/smtppassword \
  --value BPzoobfQ4cGz1PBIMMWuqVjY7BPTUxVrb/7YnJ5Gzz6V \
  --type SecureString

Testing the Container Locally

With our credentials stashed in the AWS Parameter Store, we can launch the Postfix container. Here, we are mounting our AWS credentials into the container.

docker run -v ~/.aws:/root/.aws --rm -p 25:25 --name postfix tmclnk/ses-postfix-relay

The container will verify that we have IAM credentials to access the Parameter Store. It will then retrieve our SMTP Username and SMTP Password from the Parameter Store. These were stored in /ses-relay/smtpusername and /ses-relay/smtppassword above.

With the container running (“postfix”), we can attach a shell and run a quick test.

docker exec -it postfix bash

You’ll need to paste the next part in two sections. First, launch sendmail:

sendmail -f noreply@mydomain.com success@simulator.amazonses.com

Then paste the following block of text. Note that the “.” on its own line signals sendmail to send the message.

From: MyDomain Notification
Subject: Amazon SES Test
This message was sent using Amazon SES.
.

If you haven’t verified any addresses or domains in your account’s SES configuration, you will still be able to get a sense of how this works by sending the message above.

You’ll find detailed success/failure messages in /var/log/mail.log. For example, if I use noreply@mydomain.com (which I don’t own and have not verified), then I will get a log message like

Jun 9 17:55:18 fb209921cfb7 postfix/smtp[719]: 67CB06909C: to=noreply@mydomain.com, relay=email-smtp.us-east-1.amazonaws.com[35.169.226.22]:587, delay=0.6, delays=0/0/0.54/0.06, dsn=5.0.0, status=bounced (host email-smtp.us-east-1.amazonaws.com[35.169.226.22] said: 501 Invalid MAIL FROM address provided (in reply to MAIL FROM command))

When done, you can exit the shell and kill the container with

docker kill postfix

Deploy the Container

In our specific case, we decided to bundle postfix as a sidecar container alongside our legacy applications, which expect SMTP on localhost:25. We are using Amazon ECS. To get the relay to appear on “localhost” within the Task’s virtual network, we have to use awsvpc networking mode. Our resulting task-definition.yml might look like this:

## ECS Task Definition using awsvpc networking mode
placementConstraints: []
requiresCompatibilities:
  - EC2
family: dev-myapp
taskRoleArn: arn:aws:iam::000000000000:role/ecsTaskExecutionRole
executionRoleArn: arn:aws:iam::000000000000:role/ecsTaskExecutionRole
networkMode: awsvpc
containerDefinitions:
  - name: myapp
    image: 000000000000.dkr.ecr.us-east-1.amazonaws.com/myapp:latest
  - name: ses-postfix-relay
    image: tmclnk/ses-postfix-relay:latest
    cpu: 0
    memoryReservation: 50
    essential: true
    logConfiguration:
      logDriver: awslogs
      options:
        awslogs-group: /dev/myapp
        awslogs-region: us-east-1
        awslogs-stream-prefix: myapp

If you are using Kubernetes, then simply having the container in a Pod with your application will be sufficient, as containers in a Kubernetes Pod share the same networking namespace, so every container will see the port as “localhost:25”.

Additional Configuration

On startup, the container builds a credential database for Postfix using the parameters from the parameter store. You do not need to perform those steps manually – you just need to have the credentials available in the Parameter Store.

Check the github page for information on customizing parameter store key names and configuring networking whitelists (helo_access). You are not limited to just using a local relay, but you will need to consider the risks of opening up the relay to the network, and the tradeoffs required to secure that relay, namely, TLS and authentication.

Related Links

About the Author

Object Partners profile.
Leave a Reply

Your email address will not be published.

Related Blog Posts
Natively Compiled Java on Google App Engine
Google App Engine is a platform-as-a-service product that is marketed as a way to get your applications into the cloud without necessarily knowing all of the infrastructure bits and pieces to do so. Google App […]
Building Better Data Visualization Experiences: Part 2 of 2
If you don't have a Ph.D. in data science, the raw data might be difficult to comprehend. This is where data visualization comes in.
Unleashing Feature Flags onto Kafka Consumers
Feature flags are a tool to strategically enable or disable functionality at runtime. They are often used to drive different user experiences but can also be useful in real-time data systems. In this post, we’ll […]
A security model for developers
Software security is more important than ever, but developing secure applications is more confusing than ever. TLS, mTLS, RBAC, SAML, OAUTH, OWASP, GDPR, SASL, RSA, JWT, cookie, attack vector, DDoS, firewall, VPN, security groups, exploit, […]