AWS/Terraform – Provision an EC2 instance using Infrastructure as Code

Note: Some of this is a duplicate of the AWS Lightsail article, modified for EC2, a recent Terraform 0.12.x version and accommodates the previous Lightsail implementation. If you would like to use Lightsail, please follow the IAM specific instructions in that article.

EC2 is the compute service in AWS. It is flexible, adaptable, scalable and is able to run Virtual Machine workloads to fit most every need.

In this article we will use Terraform (Infrastructure as Code) to swiftly bring up an AWS EC2 instance in us-east-1 on a static IP (Elastic IP), in a new VPC with an Internet Gateway, add a DNS Zone (Route 53) for the site in mention and install docker/docker-compose on it.

We will use ‘myweb’ as an example in this article, using the same base path of ‘dev’ that was previously created, the container-admin group (some of the IAM policy implemented there will be in use here) and using ~/.local/bin for the binary.

Please use AWS Free Tier prior to commencing with this article.

–>
Go in to the dev directory/link located within your home directory:

$ cd ~/dev

Grab Terraform:

$ wget https://releases.hashicorp.com/terraform/0.12.21/terraform_0.12.21_linux_amd64.zip

Install Unzip if you do not have it installed:

$ sudo apt update && sudo apt -y install unzip

Unzip it to ~/.local/bin and set permissions accordingly on it (type y and hit enter to replace if upgrading, at the prompt):

$ unzip terraform_0.12.21_linux_amd64.zip -d ~/.local/bin && chmod 754 ~/.local/bin/terraform

Create a work folder and change in to it:

$ mkdir -p terraform/aws/myweb/scripts && cd terraform/aws/myweb

Add an IAM Policy to the container-admin group so it will have access to EC2 and related (EIP/VPC/Routes/IGW/Route 53/SG/KeyPair):
AWS UI Console -> Services -> Security, Identity, & Compliance -> IAM -> Policies -> Create Policy -> JSON (replace <AWS ACCOUNT ID> in the Resource arn with your Account’s ID (shown under the top right drop-down (of your name) within the My Account page next to the Account Id: under Account Settings)):

 {
     "Version": "2012-10-17",
     "Statement": [
         {
             "Effect": "Allow",
             "Action": [
                 "ec2:TerminateInstances",
                 "route53:GetChange",
                 "route53:GetHostedZone",
                 "route53:ChangeTagsForResource",
                 "route53:DeleteHostedZone",
                 "route53:ListTagsForResource" 
             ],
             "Resource": [
                "arn:aws:ec2:*:<AWS ACCOUNT ID>:instance/*",
                "arn:aws:route53:::hostedzone/*",
                "arn:aws:route53:::change/*"
             ]      
         },
         {
             "Effect": "Allow",
             "Action": [
                 "ec2:DisassociateAddress",
                 "ec2:DeleteSubnet",
                 "ec2:DescribeAddresses",
                 "ec2:DescribeInstances",
                 "ec2:DescribeInstanceAttribute",
                 "ec2:CreateVpc",
                 "ec2:AttachInternetGateway",
                 "ec2:DescribeVpcAttribute",
                 "ec2:AssociateRouteTable",
                 "ec2:DescribeInternetGateways",
                 "ec2:DescribeNetworkInterfaces",
                 "ec2:CreateInternetGateway",
                 "ec2:CreateSecurityGroup",
                 "ec2:DescribeVolumes",
                 "ec2:DescribeAccountAttributes",
                 "ec2:ModifyVpcAttribute",
                 "ec2:DescribeKeyPairs",
                 "ec2:DescribeNetworkAcls",
                 "ec2:DescribeRouteTables",
                 "ec2:ReleaseAddress",
                 "ec2:ImportKeyPair",
                 "ec2:DescribeTags",
                 "ec2:DescribeVpcClassicLinkDnsSupport",
                 "ec2:CreateRouteTable",
                 "ec2:DetachInternetGateway",
                 "ec2:DisassociateRouteTable",
                 "ec2:AllocateAddress",
                 "ec2:DescribeInstanceCreditSpecifications",
                 "ec2:DescribeSecurityGroups",
                 "ec2:DescribeVpcClassicLink",
                 "ec2:DescribeImages",
                 "ec2:DescribeVpcs",
                 "ec2:DeleteVpc",
                 "ec2:AssociateAddress",
                 "ec2:CreateSubnet",
                 "ec2:DescribeSubnets",
                 "ec2:DeleteKeyPair",
                 "route53:CreateHostedZone",
                 "sts:GetCallerIdentity"
             ],
             "Resource": "*"
         }
     ]
 }

Review Policy ->

Name: AllowEC2
Description: Allow access to EC2 and related.

Create Policy.

Groups -> container-admin -> Attach Policy -> Search for AllowEC2 -> Attach Policy.

–>
Note: If you are not using Lightsail then you can disregard this section.

Edit the IAM Policy “AllowLightsail” to add an allowance to GetKeyPair in Lightsail:
AWS UI Console -> Services -> Security, Identity, & Compliance -> IAM -> Policies -> AllowLightsail -> Edit Policy -> JSON ->

Append lightsail:GetKeyPair after lightsail:DeleteKeyPair and before lightsail:GetInstance.

It will look like this:

                  "lightsail:DeleteKeyPair",
                  "lightsail:GetKeyPair",
                  "lightsail:GetInstance",

Review Policy -> Save Changes
<–

Generate an SSH Key Pair (no password) and restrict permissions on it:

$ ssh-keygen -q -t rsa -b 2048 -N '' -f ~/.ssh/myweb && chmod 400 ~/.ssh/myweb

Pin the Terraform version to greater then or equal to 0.12:

$ cat << 'EOF' > versions.tf
> terraform {
>   required_version = ">= 0.12"
> }
> EOF

Set the version to greater then or equal to 2.0 for the AWS provider, interpolate the region and use the AWS CLI credentials file:

$ cat << 'EOF' > provider.tf
> provider "aws" {
>   version                 = ">= 2.0"
>
>   region                  = var.region
>   shared_credentials_file = "~/.aws/credentials"
>   profile                 = "default"
> }
> EOF

Set the default region as a variable, set prefix of myweb and set lightsail by default to true:

$ cat << 'EOF' > vars.tf
> variable "region" {
>   default = "us-east-1"
> }
>
> variable "prefix" {
>   default = "myweb"
> }
>
> variable "lightsail" {
>   default = true
> }
> EOF

While we are here, let us create a new Lightsail script/code (if you have completed the previous AWS/Terraform against Lightsail article, then please overwrite). This will execute if no override (‘lightsail = false’) is passed. This also adds our public key as authorized (as oppose to uploading it manually as in the previous article):

$ cat << 'EOF' > lightsail.tf
> # Create a DNS Zone
> resource "aws_lightsail_domain" "myweb" {
>   count       = var.lightsail ? 1 : 0
>   domain_name = "${var.prefix}.com"
> }
>
> # Allocate a Static (Public) IP
> resource "aws_lightsail_static_ip" "myweb" {
>   count = var.lightsail ? 1 : 0
>   name  = "static-ip_${var.prefix}"
> }
>
> # Add Public Key as authorized
> resource "aws_lightsail_key_pair" "myweb" {
>   count      = var.lightsail ? 1 : 0
>   name       = var.prefix
>   public_key = file("~/.ssh/${var.prefix}.pub")
> }
>
> # Create an Ubuntu Virtual Machine with key based access and run a script on boot
> resource "aws_lightsail_instance" "myweb" {
>   count             = var.lightsail ? 1 : 0
>   name              = "site_${var.prefix}"
>   availability_zone = "${var.region}a"
>   blueprint_id      = "ubuntu_18_04"
>   bundle_id         = "micro_2_0"
>   key_pair_name     = var.prefix
>   user_data         = file("scripts/install.sh")
>
>   tags = {
>         Site = "${var.prefix}.com"
>     }
> }
>
> # Attach the Static (Public) IP
> resource "aws_lightsail_static_ip_attachment" "myweb" {
>   count          = var.lightsail ? 1 : 0
>   static_ip_name = element(aws_lightsail_static_ip.myweb[*].name, 0)
>   instance_name  = element(aws_lightsail_instance.myweb[*].name, 0)
> }
> EOF

Note: The below adds a conditional to accommodate the previous implementation against Lightsail. It will get executed when ‘lightsail = false’ is passed.

The following is performed with this script/code:

  • create a Route 53 DNS Zone of myweb.com (no A records will be added)
  • create a Virtual Private Cloud for network 10.0.0.0/16 (tenancy is default)
  • add a subnet of 10.0.1.0/24 within the VPC
  • allocate a static Public IP
  • create a Security Group and add a Security rule for allowing SSH (port 22) Inbound
  • create an Internet Gateway and add a route out to it
  • create a T3a.micro instance (tenancy is default) based off of Ubuntu 18_04, our public key added as authorized and reference an extraneous file for user_data (initialization script on Virtual Machine boot). Elastic/Root Block Store is GP2
  • DNS support is enabled but DNS host names is not
  • tag all resources

Note: vpc_security_group_ids is used as oppose to security_groups, as the latter would cause a destroy/create of the instance every time an apply is performed:

$ cat << 'EOF' > ec2.tf
> # Create a DNS Zone 
> resource "aws_route53_zone" "myweb" {
>   count   = var.lightsail ? 0 : 1
>   name    = "${var.prefix}.com"
>   comment = "${var.prefix}.com (Public)"
>
>   tags = {
>         Site = "${var.prefix}.com"
>         Name = "${var.prefix}-dn"
>     }
> }
>
> # Create a Security Group and allow inbound port(s)
> resource "aws_security_group" "myweb" {
>   count       = var.lightsail ? 0 : 1
>   name        = var.prefix
>   description = "Allow Ports"
>   vpc_id      = element(aws_vpc.myweb[*].id, 0)
>
>   ingress {
>        from_port   = 22
>        to_port     = 22
>        protocol    = "tcp"
>        cidr_blocks = ["0.0.0.0/0"]
>        description = "SSH"
>     }
>
>   egress {
>        from_port   = 0
>        to_port     = 0
>        protocol    = "-1"
>        cidr_blocks = ["0.0.0.0/0"]
>        description = "All"
>     }
>
>   tags = {
>         Site = "${var.prefix}.com"
>         Name = "${var.prefix}-sg"
>     }
> }
>
> # Create a Virtual Private Cloud
> resource "aws_vpc" "myweb" {
>   count            = var.lightsail ? 0 : 1
>   cidr_block       = "10.0.0.0/16"
>   instance_tenancy = "default"
>
>   tags = {
>         Site = "${var.prefix}.com"
>         Name = "${var.prefix}-vpc"
>     }
> }
>
> # Add a Subnet
> resource "aws_subnet" "internal" {
>   count             = var.lightsail ? 0 : 1
>   vpc_id            = element(aws_vpc.myweb[*].id, 0)
>   cidr_block        = "10.0.1.0/24"
>   availability_zone = "${var.region}a"
>
>   tags = {
>         Site = "${var.prefix}.com"
>         Name = "internal"
>     }
> }
>
> # Create an Internet Gateway
> resource "aws_internet_gateway" "myweb" {
>   count  = var.lightsail ? 0 : 1
>   vpc_id = element(aws_vpc.myweb[*].id, 0)
>
>   tags = {
>         Site = "${var.prefix}.com"
>         Name = "${var.prefix}-igw"
>     }
> }
>
> # Allocate a Static Public IP
> resource "aws_eip" "external" {
>   count             = var.lightsail ? 0 : 1
>   vpc               = true
>   instance          = element(aws_instance.myweb[*].id, 0)
>   depends_on        = [aws_internet_gateway.myweb]
>
>   tags = {
>         Site = "${var.prefix}.com"
>         Name = "external"
>     }
> }
>
> # Add a route to the Internet Gateway
> resource "aws_route_table" "myweb" {
>   count  = var.lightsail ? 0 : 1
>   vpc_id = element(aws_vpc.myweb[*].id, 0)
>
>   route {
>        cidr_block = "0.0.0.0/0"
>        gateway_id = element(aws_internet_gateway.myweb[*].id, 0)
>     }
>
>   tags = {
>         Site = "${var.prefix}.com"
>         Name = "${var.prefix}-rt"
>     }
> }
>
> # Associate the route table with the Subnet
> resource "aws_route_table_association" "myweb" {
>   count          = var.lightsail ? 0 : 1
>   subnet_id      = element(aws_subnet.internal[*].id, 0)
>   route_table_id = element(aws_route_table.myweb[*].id, 0)
> }
>
> # Add Public Key as authorized
> resource "aws_key_pair" "myweb" {
>   count      = var.lightsail ? 0 : 1
>   key_name   = var.prefix
>   public_key = file("~/.ssh/${var.prefix}.pub")
> }
>
> # Select Ubuntu 18.04
> data "aws_ami" "ubuntu" {
>   count       = var.lightsail ? 0 : 1
>   most_recent = true
>
>   filter {
>         name   = "name"
>         values = ["ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-*"]
>     }
>
>   filter {
>         name   = "virtualization-type"
>         values = ["hvm"]
>     }
>
>   owners = ["099720109477"] # Canonical
> }
>
> # Create an Ubuntu Virtual Machine with key based access and run a script on boot
> resource "aws_instance" "myweb" {
>   count                    = var.lightsail ? 0 : 1
>   ami                      = element(data.aws_ami.ubuntu[*].id, 0)
>   instance_type            = "t3a.micro"
>   availability_zone        = "${var.region}a"
>   key_name                 = var.prefix
>   vpc_security_group_ids   = [element(concat(aws_security_group.myweb[*].id, list("")), 0)]
>   user_data                = file("scripts/install.sh")
>   subnet_id                = element(aws_subnet.internal[*].id, 0)
>   tenancy                  = "default"
>
>   tags = {
>         Site = "${var.prefix}.com"
>         Name = "${var.prefix}-ec2"
>     }
> } 
> EOF

Output our allocated and attached static Public IP after creation. Also output an inventory file to the Ansible workarea for later consumption and accommodate our previous Lightsail implementation:

$ cat << 'EOF' > output.tf
> output "static_public_ip" {
>   value = var.lightsail ? element(aws_lightsail_static_ip.myweb[*].ip_address, 0) : element(aws_eip.external[*].public_ip, 0)
> }
>
> resource "local_file" "hosts" {
>   content              = "[vps]\n${var.lightsail ? element(aws_lightsail_static_ip.myweb[*].ip_address, 0) : element(aws_eip.external[*].public_ip, 0)} ansible_connection=ssh ansible_user=ubuntu ansible_ssh_private_key_file=~/.ssh/${var.prefix} instance=${var.lightsail ? element(aws_lightsail_instance.myweb[*].name, 0) : element(aws_instance.myweb[*].tags["Name"], 0)}"
>   filename             = pathexpand("~/dev/ansible/hosts-aws")
>   directory_permission = 0754
>   file_permission      = 0664
> }
> EOF

If you have gone through the AWS/Terraform against Lightsail article, then please delete the template file for the install script:

$ rm install.tf

Create the shell script for user_data:

$ cat << 'EOF' > scripts/install.sh
> #!/bin/bash
>
> MY_HOME="/home/ubuntu"
> export DEBIAN_FRONTEND=noninteractive
>
> # Install prereqs
> apt update
> apt install -y python3-pip apt-transport-https ca-certificates curl software-properties-common
> # Install docker
> curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
> add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
> apt update
> apt install -y docker-ce
> # Install docker-compose
> su ubuntu -c "mkdir -p $MY_HOME/.local/bin" 
> su ubuntu -c "pip3 install docker-compose --upgrade --user && chmod 754 $MY_HOME/.local/bin/docker-compose"
> usermod -aG docker ubuntu
> # Add PATH
> printf "\nexport PATH=\$PATH:$MY_HOME/.local/bin\n" >> $MY_HOME/.bashrc
>
> exit 0
> EOF

Initialize the directory:

$ terraform init

Run a dry-run to see what will occur:

$ terraform plan -var 'lightsail=false'

Provision:

$ terraform apply -var 'lightsail=false' -auto-approve

Log on to the instance after a short while:

$ ssh -i ~/.ssh/myweb ubuntu@<The value of static_public_ip that was reported.  One can also use 'terraform output static_public_ip' to print it again.>

Type yes and hit enter to accept.

On the host (a short while is needed for the run-once script to complete):

$ docker --version
$ docker-compose --version
$ logout

Tear down what was created by first performing a dry-run to see what will occur:

$ terraform plan -var 'lightsail=false' -destroy 

Tear down the instance:

$ terraform destroy -var 'lightsail=false' -auto-approve

<–

References:

Source:
terraform_aws_myweb