Custom email domain using Gmail, Cloudflare, and Amazon SES

This article describes how to set up email for a custom domain to use Gmail as mail client and for storage, CloudFlare for DNS management and for incoming mail redirects, and Amazon SES for sending outgoing mail. Originally I've set it up manually, and later wrote a Terraform module which configures everything automatically: terraform-aws-ses-cloudflare-mail-setup.

Why Use a Custom Domain, but not Own a Mail Server?

Changing an email address is hard: email is still the main identity in multiple internet services, and gets thousands of references everywhere over the years. Using an email address on a custom domain reduces the risk of being forced to change the email address everywhere if an email service provider disappears, if the account gets blocked by the email service provider, or if the access to the account gets lost for any other reason.

Hosting a full email setup is also hard and full of struggle: open source web mail clients are terrible compared to Gmail; even most desktop mail clients are bad compared to Gmail. Anti-spam systems hate small mail servers hosted on cheap VPSes. A dozen of things needs to be configured properly to cooperate, and if something goes wrong, the messages just get dropped (either by the setup, or by the receiver). There are some shortcuts (r/selfhosted references some Docker images which are relatively easy to use; there are web hosting control panels such as ISPmanager that can configure mail and other servers), but I am not ready to have surprising debugging adventures when my mail suddenly stops working.

There are services which provide hosted mail for the domain, but they are either prohibitively expensive (like Google Workspace, which I used for 17 years and decided to leave), or much worse than Gmail (like any random mail hosting that creates a mailbox with POP3/IMAP/SMTP credentials and calls it a day). I tried a couple of services (Proton and Yandex), but still wasn't happy with the quality.

Using Gmail as a backend for an address on a custom domain is a good middle ground: there is no strong dependency on Gmail forever, but all benefits of Gmail can still be used. Gmail has good support for email aliases, including the ones on custom domains: it can show which alias the email came to, and can respond from the same address. So start with creating a Gmail address, if you don't have one.

Incoming Email via Cloudflare

Cloudflare is a cloud provider offering a variety of services. Five of them are relevant for hosting email:

• Registrar is a very cheap domain registration service which supports a variety of top level domains. There is a small caveat: if a domain is registered through Cloudflare, it must use
• DNS is a free authoritative DNS hosting. It serves DNS records and provides a web UI and an API
• to manage these records.
• Email Routing is a free mail redirect service. It is a mail server which accepts mail for a custom domain, looks up a redirect rule, and forwards the messages to another address not related to the domain or to Cloudflare.
• DMARC Management is a free service that ingests authentication reports for emails originating from the domain and displays whether there were DKIM or SPF validation failures for messages.
• Pages is a free static content web hosting. It can be used to put some information on the domain, if the domain does not need a proper web site.

Cloudflare has somewhat unintuitive split between what's managed on the main account dashboard and what's managed on a domain-specific dashboard. Domain registration happens on the top level of the account. After the domain is registered, it appears in the "Websites" list of the top level dashboard, and all further management DNS and mail redirect management happens there. But hosting static content requires creating a Pages application in the top level dashboard again, and then pointing the website to this new Pages application.

The configuration is almost trivial: register a domain (or create a website on the free plan), enable DMARC management, go to email routing settings, add the target Gmail addresses as destination addresses, and set up routing rules to redirect incoming mail from addresses on the custom domain to these Gmail addresses. Cloudflare even configures DNS for mail redirects automatically. There is nothing to customize and everything works out of the box. The total cost of the Cloudflare setup is equal to the total cost of the domain registration, and is around 10$ per year.

Outgoing Email

Sending outgoing messages is much more complicated: spam detectors of recipients are very sensitive to misconfiguration; mail servers have reputation in spam detectors, so email service providers try very hard to prevent outgoing spam or misconfigured messages. Gmail, for example, sends messages only from addresses hosted on Gmail, and requires an external SMTP server for custom domains.

Most paid SMTP services are newsletters- and business-oriented: they are optimized for mass delivery and tracking conversion, or at least for transactional emails of websites, such as password reset emails. Many services store a lot of data about messages passing through them: some store headers, some even store bodies.

I have picked Amazon Simple Email Service (SES): Amazon is a large company which should handle the data well. It has a very low price for email -- 0.10$ for 1000 emails -- but a very high price for a dedicated IP address -- 25$ per month. However, the internet says that the deliverability is lower than their competitors have.

Two other services, Postmark and SMTP2GO, made it to the short list of my options. I will try them if I have any problems with Amazon. Their pricing model is package-based: they give the first 100 or 1000 outgoing messages per month for free, and then bill 10$ or 15$ per month for 10000 messages per month. They also require filing a form with use case description to get approved; both of them approved my pattern of usage within hours, but Postmark required a round-trip email with more explanations.

Amazon SES

Amazon AWS is also a cloud provider offering a much wider variety of services. One of these services is Amazon SES: a cheap service for sending outgoing emails from custom domains, which supports sending through SMTP. Unlike Cloudflare, it is regionalised: all configuration is specific to an AWS region, and there are some limitations on what is available in what region (for example, Milan does not support SMTP, which is needed for the setup described here). After registration the console redirected me to Stockholm (eu-north-1), so I stayed in it for no particular reason.

Delivery Errors and Configuration Sets

It's important to be notified when something goes wrong with email delivery. SNS is tuned for mass email sending, so it doesn't send bounce emails to the senders: instead, it reports them programmatically as structured events. Amazon Simple Notification Service (SNS) can forward these events back to email messages. SES has a built-in configuration option to forward bounces to the senders as emails, but it didn't work in my account, and I don't have a support subscription to ask a question about it Update(2024-05-24): now it seems to work, but the forwarded error messages contain less information than the SNS reports, so I've decided to continue using SNS.

Configuration set is a blueprint for some delivery configuration that SES will apply to outgoing messages. The important part of configuration is where delivery failure notifications will be sent. Go to SES console and create a configuration set. I am creating one configuration set per redirect recipient, so that failure notifications can be delivered to the sender. Another option is creating the only one and manually dealing with delivery errors.

Configuring a configuration set requires creating it, creating a destination SNS topic, subscribing the topic to notifications from the configuration set, and subscribing an email address to notifications from the topic.

Identities and Domain Validation

The next step is creating identities: domains and addresses on which behalf SES will send messages. Start with creating a domain, and then create all email addresses in this domain that will use SES for outgoing email. The following configuration fields are important and must be set for both domain and email addresses:

• Assign a default configuration set: set it to the just created configuration set to ensure failure notification delivery.

• Use a custom MAIL FROM domain: enable it and set it to some subdomain of the main domain which will not be used for other purposes. I used mail. as the subdomain.

  Using the default MAIL FROM domain can result in some mail clients (including Outlook) mentioning via amazonses.com in the From field and increase the probability of messages going to spam, especially if transitioning an existing domain to SES.

• Easy DKIM: enable it, so SES will manage encryption keys and DNS records that confirm that SES is authorized to send emails on behalf of the identity.

After domain identity creation, SES will display the DNS records that the domain must have to send mail and pass the validation. Add these records to CloudFlare, and turn off proxying by setting Proxy status to DNS only.

After mail address identity creation, SES will send a verification link to this email address. If everything in Cloudflare is configured correctly, the link will arrive to the Gmail address using the just set up email redirect. This identity also has DNS record requirements, but the records are the same as for the domain identity, so they should be green automatically.

Sandbox

Right after signing up, the account is sandboxed: it can send only few emails and only between verified identities. To get the account unblocked, click "Request production access" and fill a form describing the use case. I wrote that I use SNS for personal email and expect to send at most 20 messages per day; after a two days long review, my account got unsandboxed and I was allowed to send 50000 messages per day.

SMTP Accounts

The final step of the set up is creating SMTP accounts, so that Gmail and other mail clients can use SES to send mail. Go to the SMTP settings tab, click Create SMTP credentials, come up with a user name (which will be visible only in logs and in the AWS console), click Create, and write down the presented SMTP user name and SMTP password (these won't be viewable anymore after closing the page). Click Return to the SES console and write down the SMTP endpoint address.

Now go to Gmail or any other mail client and enter these SMTP credentials for outgoing mail. In Gmail this is done when adding an email alias in Settings, Accounts and Import tab, Send mail as section.

By default, SMTP credentials can send messages from any identity that is added to the account, and it's fine to keep the config as it is if all senders are trusted.

🎉 Now everything should work! 🎉

Checking DMARC Reports in Cloudflare

After using the setup for a couple of days, go to Cloudflare DMARC Management and check sources of messages. The ones coming from Amazon should have 100% rates of DMARC pass and DKIM aligned. If there are failures, ensure that the DNS records in Cloudflare are correct.

SPF alignment for emails sent using SES is not possible, but that's fine, as DKIM is enough to pass DMARC checks. After clicking on the Amazon source, check that the "Envelop from: domain" value is a subdomain of your domain and is not amazonses.com, as it might affect spam classification.

Afterword

This article describes the email setup that I want to use for personal email for multiple years. I have configured it in May 2024 for the first time, so I don't fully know all trade-offs: for example, what is the actual Amazon SES deliverability for personal messages and how it is affected by Amazon SES not supporting SPF alignment.

Configuring Amazon for this purpose feels surprisingly hard: I had to start a blog and write an article to ensure that I don't forget how it works in a year. If I will have to do it again, I will probably write some automation. Update(2024-05-24): My only commenter on Reddit pointed out that automation is already written and is called CDK (or, more generally, any infrastructure as a code tool works for this). I don't understand why, but somehow it didn't occur to me before this comment to use Terraform to automate the setup. Learning Terraform was fun and I published the module on Github: terraform-aws-ses-cloudflare-mail-setup.

Now I am going to the next challenge: safely migrating my custom domain from Google Workspace to this setup, while preserving the data and having no email disruptions. This is described in the next article: Escaping workspace (pt. 1)