Introduction

Amazon RDS is a relational database service available on Amazon Web Services. It is essentially a managed database server with a volume for the data. Both cost money. Having a large dataset is not the only reason have big volume: the volume size determines the number of I/O operations (IOPS) the volume can do. If you have high load on your RDS instance you may very well run out of IOPS which can result in a disaster. Scaling up the volume size is easy to avoid such disasters, but it is much more difficult to reduce RDS storage size. This is because downsizing RDS storage size is not officially supported by AWS. If the load on your RDS instance goes down (e.g. due to refactoring), you may end up having a database with an oversized volume that costs lots of money for no reason.

It is trivial to downsize an AWS RDS instance. It is equally trivial to upsize AWS RDS volumes. However, to downsize AWS RDS volume you need to jump through lots of hoops.

So, why would you ever want to downsize an AWS RDS volume? One reason might be that you increased the volume size simply to get more burst credits for the volume. Then you did refactoring that reduced the need to have that many credits and want to save money. Or maybe you genuinely had a huge amount of data on your RDS instance volume, removed some of it but got stuck with a huge volume.

Let’s Encrypt on AWS can be a can of worms if you don’t know what you’re doing. Photo credit.

Introduction

Let’s Encrypt and the ACME protocol enable getting free (as in beer) TLS/SSL certificates. The main use for Let’s Encrypt is to enable HTTPS on web servers. Let’s Encrypt TLS certificates expire pretty quickly, in 3 months, but the idea is that renewal is automated so that you don’t ever really have to worry about expiration, unlike with commercial TLS certificates, which you typically need to manually renew every 12 months. Using Let’s Encrypt on AWS is in some cases a very viable option, but not so much in others. In many cases trying to use Let’s Encrypt certificates leads you down the path of Lovecraftian horrors and madness. Because “it all depends” I’ll try to explain when using Let’s Encrypt is viable in the AWS context and when you really should just give up an use certificates generated by the Amazon Certificate Manager (“ACM”) instead.

The symptoms

I recently received and email from Amazon Route53 asking me to verify the contact details for one of our registered DNS domains (replaced here with example.org). And sure enough, I saw wrong Route53 domain contact details in the email:

Subject: Confirm that contact information for example.org is correct.
Message body:

Dear AWS customer,

This message is a reminder to help you keep the contact information for your domain registration up to date. WHOIS records include the following information:

  Domains:
  example.org

  Registrant:
  Name: On behalf of example.org owner
  Organization Name: Identity Protection Service
  Address: PO Box 786
  City: Hayes
  State/Province: Middlesex
  Country: GB
  Postal Code: UB3 9TR

  Administrative Contact:
  Name: On behalf of example.org owner
  Organization Name: Identity Protection Service
  Address: PO Box 786
  City: Hayes
  State/Province: Middlesex
  Country: GB
  Postal Code: UB3 9TR
  Phone: +44.1483307527
  Fax: +44.1483304031
  Email: e793d865-97ab-42ab-805c-50cc36fd9202@identity-protect.org

  Technical Contact:
  Name: On behalf of example.org owner
  Organization Name: Identity Protection Service
  Address: PO Box 786
  City: Hayes
  State/Province: Middlesex
  Country: GB
  Postal Code: UB3 9TR
  Phone: +44.1483307527
  Fax: +44.1483304031
  Email: e793d865-97ab-42ab-805c-50cc36fd9202@identity-protect.org

The contact information provided above is what is shown in the public WHOIS. If your domain is privacy protected, the information will be different than the contact data you have submitted. To verify your contact information, please open the Amazon Route 53 console to view your current contact information and make any necessary corrections:
https://console.aws.amazon.com/route53/home#DomainDetail:example.org

If your information is accurate, you do not need to take any action.

Important
Under the terms of your registration agreement, providing false contact data can be grounds for the cancellation of your domain name registration.

Regards,
Amazon Route 53 

These wrong Route53 domain contact details originate from whois data for the registered domain. One might reasonably expect the whois data to match the contact details for AWS Route53 domain. However, in this case the Route 53 contact details were correct, but whois data seemed to be out of sync with those.

Causes for the Terraform AWS UnauthorizedOperation errors

Terraform is an infrastructure as code tool you can use to configure Cloud resources in AWS. When using Terraform AWS provider you frequently run into various UnauthorizedOperation errors when creating, modifying or deleting resources. That happens unless you do what you should not do and let Terraform use an AWS API key tied to an IAM user with full admin privileges. In almost all cases Terraform UnauthorizedOperation errors are caused by missing IAM permissions. It can sometimes be a bit tedious to get to the root cause, because Terraform may not show it in its normal output.

Introduction

AWS recommends that you disable S3 bucket ACLs for all new buckets. To understand why some background information is needed.

AWS S3 providers two ways to manage access to S3 buckets and objects:

• Access policies: these are standard IAM policies that you attach to users (“user policies”) or resources (“resource-based policies”). IAM policies are the recommended method for managing access to Amazon S3 nowadays.

• Access Control Lists (ACLs): ACLs are an older access control method for AWS S3. ACL map to IAM permissions. ACLs works on the bucket and object (file) file.

When you create a distribution, AWS creates several DNS A records with the same name (e.g. d25gma2ea3ckma.cloudfront.net) which point to IPs the distribution is using. Then, typically, you would define CNAME(s) pointing to that cloudfront.net address in your own DNS. Each Cloudfront distribution has a list of aliases, similar to Subject Alternative Names (“SAN”) in SSL certificates. The aliases should match the CNAME(s) you’ve set in DNS.

Now, this is all good until you need to migrate those Cloudfront distributions to, say, a new AWS account. If you attempt to create a new distribution (here with Terraform) with the same alias you will get this error:

The aws_instance resource in Terraform can automatically create the default network interface for you. There are cases, however, when you notice that the default network interface is not enough anymore, and modifying it via the limited aws_instance parameters is not sufficient. In these cases you can convert the interface into an aws_network_interface resource, but the process is far from straightforward.

The first step is to create a Terraform resource for the network interface:

In AWS EBS (“Elastic Block Storage”) is the underlying technology that (virtual) hard disks of your instances (virtual machines) use. You can take snapshots of those virtual hard disks and use those snapshots to, for example:

• Debugging issues with unbootable virtual machines: attach and then mount the snapshot on another virtual machine and investigate what is wrong.
• Creating AMI images from which you can launch new instances
• Sharing EBS volumes (between regions and AWS accounts), e.g. for purposes of migrating VMs

Here we’ll focus on the last use-case: being able to create copies of virtual machines on another AWS account. The reason why I even bothered writing this blog post is that most of the articles on the Internet do not cover this use-case: they assume you’re working within one AWS account and/or one region. The use-case covered here requires a few extra steps:

This article assumes that the user backend for Keycloak is FreeIPA. Regardless of that the instructions will apply to any other setup with minor modifications. Here we use two different AWS accounts renamed to 123412341234 and 567856785678 to protect the personal information of the innocent. The Keycloak staging cluster on which this integration was done is called id-staging.example.org and the production cluster is called id.example.org. The Keycloak realm is EXAMPLE.ORG. Replace those with the values correct for your environment.