Note: Some of this is a duplicate of the AWS Lightsail article, modified for EC2, a recent Terraform 0.12.x version and accommodates the previous Lightsail implementation. If you would like to use Lightsail, please follow the IAM specific instructions in that article.
EC2 is the compute service in AWS. It is flexible, adaptable, scalable and is able to run Virtual Machine workloads to fit most every need.
In this article we will use Terraform (Infrastructure as Code) to swiftly bring up an AWS EC2 instance in us-east-1 on a static IP (Elastic IP), in a new VPC with an Internet Gateway, add a DNS Zone (Route 53) for the site in mention and install docker/docker-compose on it.
We will use ‘myweb’ as an example in this article, using the same base path of ‘dev’ that was previously created, the container-admin group (some of the IAM policy implemented there will be in use here) and using ~/.local/bin for the binary.
Please use ‘AWS Free Tier‘ prior to commencing with this article.
–>
Go in to the dev directory/link located within your home directory:
$ cd ~/dev
Grab Terraform:
$ wget https://releases.hashicorp.com/terraform/0.12.21/terraform_0.12.21_linux_amd64.zip
Install Unzip if you do not have it installed:
$ sudo apt update && sudo apt -y install unzip
Unzip it to ~/.local/bin and set permissions accordingly on it (type y and hit enter to replace if upgrading, at the prompt):
$ unzip terraform_0.12.21_linux_amd64.zip -d ~/.local/bin && chmod 754 ~/.local/bin/terraform
Create a work folder and change in to it:
$ mkdir -p terraform/aws/myweb/scripts && cd terraform/aws/myweb
Add an IAM Policy to the container-admin group so it will have access to EC2 and related (EIP/VPC/Routes/IGW/Route 53/SG/KeyPair):
AWS UI Console -> Services -> Security, Identity, & Compliance -> IAM -> Policies -> Create Policy -> JSON (replace <AWS ACCOUNT ID> in the Resource arn with your Account’s ID (shown under the top right drop-down (of your name) within the My Account page next to the Account Id: under Account Settings)):
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:TerminateInstances",
"route53:GetChange",
"route53:GetHostedZone",
"route53:ChangeTagsForResource",
"route53:DeleteHostedZone",
"route53:ListTagsForResource"
],
"Resource": [
"arn:aws:ec2:*:<AWS ACCOUNT ID>:instance/*",
"arn:aws:route53:::hostedzone/*",
"arn:aws:route53:::change/*"
]
},
{
"Effect": "Allow",
"Action": [
"ec2:DisassociateAddress",
"ec2:DeleteSubnet",
"ec2:DescribeAddresses",
"ec2:DescribeInstances",
"ec2:DescribeInstanceAttribute",
"ec2:CreateVpc",
"ec2:AttachInternetGateway",
"ec2:DescribeVpcAttribute",
"ec2:AssociateRouteTable",
"ec2:DescribeInternetGateways",
"ec2:DescribeNetworkInterfaces",
"ec2:CreateInternetGateway",
"ec2:CreateSecurityGroup",
"ec2:DescribeVolumes",
"ec2:DescribeAccountAttributes",
"ec2:ModifyVpcAttribute",
"ec2:DescribeKeyPairs",
"ec2:DescribeNetworkAcls",
"ec2:DescribeRouteTables",
"ec2:ReleaseAddress",
"ec2:ImportKeyPair",
"ec2:DescribeTags",
"ec2:DescribeVpcClassicLinkDnsSupport",
"ec2:CreateRouteTable",
"ec2:DetachInternetGateway",
"ec2:DisassociateRouteTable",
"ec2:AllocateAddress",
"ec2:DescribeInstanceCreditSpecifications",
"ec2:DescribeSecurityGroups",
"ec2:DescribeVpcClassicLink",
"ec2:DescribeImages",
"ec2:DescribeVpcs",
"ec2:DeleteVpc",
"ec2:AssociateAddress",
"ec2:CreateSubnet",
"ec2:DescribeSubnets",
"ec2:DeleteKeyPair",
"route53:CreateHostedZone",
"sts:GetCallerIdentity"
],
"Resource": "*"
}
]
}
Review Policy ->
Name: AllowEC2
Description: Allow access to EC2 and related.
Create Policy.
Groups -> container-admin -> Attach Policy -> Search for AllowEC2 -> Attach Policy.
–>
Note: If you are not using Lightsail then you can disregard this section.
Edit the IAM Policy “AllowLightsail” to add an allowance to GetKeyPair in Lightsail:
AWS UI Console -> Services -> Security, Identity, & Compliance -> IAM -> Policies -> AllowLightsail -> Edit Policy -> JSON ->
Append lightsail:GetKeyPair after lightsail:DeleteKeyPair and before lightsail:GetInstance.
It will look like this:
"lightsail:DeleteKeyPair",
"lightsail:GetKeyPair",
"lightsail:GetInstance",
Review Policy -> Save Changes
<–
Generate an SSH Key Pair (no password) and restrict permissions on it:
$ ssh-keygen -q -t rsa -b 2048 -N '' -f ~/.ssh/myweb && chmod 400 ~/.ssh/myweb
Pin the Terraform version to greater then or equal to 0.12:
$ cat << 'EOF' > versions.tf
> terraform {
> required_version = ">= 0.12"
> }
> EOF
Set the version to greater then or equal to 2.0 for the AWS provider, interpolate the region and use the AWS CLI credentials file:
$ cat << 'EOF' > provider.tf
> provider "aws" {
> version = ">= 2.0"
>
> region = var.region
> shared_credentials_file = "~/.aws/credentials"
> profile = "default"
> }
> EOF
Set the default region as a variable, set prefix of myweb and set lightsail by default to true:
$ cat << 'EOF' > vars.tf
> variable "region" {
> default = "us-east-1"
> }
>
> variable "prefix" {
> default = "myweb"
> }
>
> variable "lightsail" {
> default = true
> }
> EOF
While we are here, let us create a new Lightsail script/code (if you have completed the previous AWS/Terraform against Lightsail article, then please overwrite). This will execute if no override (‘lightsail = false’) is passed. This also adds our public key as authorized (as oppose to uploading it manually as in the previous article):
$ cat << 'EOF' > lightsail.tf
> # Create a DNS Zone
> resource "aws_lightsail_domain" "myweb" {
> count = var.lightsail ? 1 : 0
> domain_name = "${var.prefix}.com"
> }
>
> # Allocate a Static (Public) IP
> resource "aws_lightsail_static_ip" "myweb" {
> count = var.lightsail ? 1 : 0
> name = "static-ip_${var.prefix}"
> }
>
> # Add Public Key as authorized
> resource "aws_lightsail_key_pair" "myweb" {
> count = var.lightsail ? 1 : 0
> name = var.prefix
> public_key = file("~/.ssh/${var.prefix}.pub")
> }
>
> # Create an Ubuntu Virtual Machine with key based access and run a script on boot
> resource "aws_lightsail_instance" "myweb" {
> count = var.lightsail ? 1 : 0
> name = "site_${var.prefix}"
> availability_zone = "${var.region}a"
> blueprint_id = "ubuntu_18_04"
> bundle_id = "micro_2_0"
> key_pair_name = var.prefix
> user_data = file("scripts/install.sh")
>
> tags = {
> Site = "${var.prefix}.com"
> }
> }
>
> # Attach the Static (Public) IP
> resource "aws_lightsail_static_ip_attachment" "myweb" {
> count = var.lightsail ? 1 : 0
> static_ip_name = element(aws_lightsail_static_ip.myweb[*].name, 0)
> instance_name = element(aws_lightsail_instance.myweb[*].name, 0)
> }
> EOF
Note: The below adds a conditional to accommodate the previous implementation against Lightsail. It will get executed when ‘lightsail = false’ is passed.
The following is performed with this script/code:
- create a Route 53 DNS Zone of myweb.com (no A records will be added)
- create a Virtual Private Cloud for network 10.0.0.0/16 (tenancy is default)
- add a subnet of 10.0.1.0/24 within the VPC
- allocate a static Public IP
- create a Security Group and add a Security rule for allowing SSH (port 22) Inbound
- create an Internet Gateway and add a route out to it
- create a T3a.micro instance (tenancy is default) based off of Ubuntu 18_04, our public key added as authorized and reference an extraneous file for user_data (initialization script on Virtual Machine boot). Elastic/Root Block Store is GP2
- DNS support is enabled but DNS host names is not
- tag all resources
Note: vpc_security_group_ids is used as oppose to security_groups, as the latter would cause a destroy/create of the instance every time an apply is performed:
$ cat << 'EOF' > ec2.tf
> # Create a DNS Zone
> resource "aws_route53_zone" "myweb" {
> count = var.lightsail ? 0 : 1
> name = "${var.prefix}.com"
> comment = "${var.prefix}.com (Public)"
>
> tags = {
> Site = "${var.prefix}.com"
> Name = "${var.prefix}-dn"
> }
> }
>
> # Create a Security Group and allow inbound port(s)
> resource "aws_security_group" "myweb" {
> count = var.lightsail ? 0 : 1
> name = var.prefix
> description = "Allow Ports"
> vpc_id = element(aws_vpc.myweb[*].id, 0)
>
> ingress {
> from_port = 22
> to_port = 22
> protocol = "tcp"
> cidr_blocks = ["0.0.0.0/0"]
> description = "SSH"
> }
>
> egress {
> from_port = 0
> to_port = 0
> protocol = "-1"
> cidr_blocks = ["0.0.0.0/0"]
> description = "All"
> }
>
> tags = {
> Site = "${var.prefix}.com"
> Name = "${var.prefix}-sg"
> }
> }
>
> # Create a Virtual Private Cloud
> resource "aws_vpc" "myweb" {
> count = var.lightsail ? 0 : 1
> cidr_block = "10.0.0.0/16"
> instance_tenancy = "default"
>
> tags = {
> Site = "${var.prefix}.com"
> Name = "${var.prefix}-vpc"
> }
> }
>
> # Add a Subnet
> resource "aws_subnet" "internal" {
> count = var.lightsail ? 0 : 1
> vpc_id = element(aws_vpc.myweb[*].id, 0)
> cidr_block = "10.0.1.0/24"
> availability_zone = "${var.region}a"
>
> tags = {
> Site = "${var.prefix}.com"
> Name = "internal"
> }
> }
>
> # Create an Internet Gateway
> resource "aws_internet_gateway" "myweb" {
> count = var.lightsail ? 0 : 1
> vpc_id = element(aws_vpc.myweb[*].id, 0)
>
> tags = {
> Site = "${var.prefix}.com"
> Name = "${var.prefix}-igw"
> }
> }
>
> # Allocate a Static Public IP
> resource "aws_eip" "external" {
> count = var.lightsail ? 0 : 1
> vpc = true
> instance = element(aws_instance.myweb[*].id, 0)
> depends_on = [aws_internet_gateway.myweb]
>
> tags = {
> Site = "${var.prefix}.com"
> Name = "external"
> }
> }
>
> # Add a route to the Internet Gateway
> resource "aws_route_table" "myweb" {
> count = var.lightsail ? 0 : 1
> vpc_id = element(aws_vpc.myweb[*].id, 0)
>
> route {
> cidr_block = "0.0.0.0/0"
> gateway_id = element(aws_internet_gateway.myweb[*].id, 0)
> }
>
> tags = {
> Site = "${var.prefix}.com"
> Name = "${var.prefix}-rt"
> }
> }
>
> # Associate the route table with the Subnet
> resource "aws_route_table_association" "myweb" {
> count = var.lightsail ? 0 : 1
> subnet_id = element(aws_subnet.internal[*].id, 0)
> route_table_id = element(aws_route_table.myweb[*].id, 0)
> }
>
> # Add Public Key as authorized
> resource "aws_key_pair" "myweb" {
> count = var.lightsail ? 0 : 1
> key_name = var.prefix
> public_key = file("~/.ssh/${var.prefix}.pub")
> }
>
> # Select Ubuntu 18.04
> data "aws_ami" "ubuntu" {
> count = var.lightsail ? 0 : 1
> most_recent = true
>
> filter {
> name = "name"
> values = ["ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-*"]
> }
>
> filter {
> name = "virtualization-type"
> values = ["hvm"]
> }
>
> owners = ["099720109477"] # Canonical
> }
>
> # Create an Ubuntu Virtual Machine with key based access and run a script on boot
> resource "aws_instance" "myweb" {
> count = var.lightsail ? 0 : 1
> ami = element(data.aws_ami.ubuntu[*].id, 0)
> instance_type = "t3a.micro"
> availability_zone = "${var.region}a"
> key_name = var.prefix
> vpc_security_group_ids = [element(concat(aws_security_group.myweb[*].id, list("")), 0)]
> user_data = file("scripts/install.sh")
> subnet_id = element(aws_subnet.internal[*].id, 0)
> tenancy = "default"
>
> tags = {
> Site = "${var.prefix}.com"
> Name = "${var.prefix}-ec2"
> }
> }
> EOF
Output our allocated and attached static Public IP after creation. Also output an inventory file to the Ansible workarea for later consumption and accommodate our previous Lightsail implementation:
$ cat << 'EOF' > output.tf
> output "static_public_ip" {
> value = var.lightsail ? element(aws_lightsail_static_ip.myweb[*].ip_address, 0) : element(aws_eip.external[*].public_ip, 0)
> }
>
> resource "local_file" "hosts" {
> content = "[vps]\n${var.lightsail ? element(aws_lightsail_static_ip.myweb[*].ip_address, 0) : element(aws_eip.external[*].public_ip, 0)} ansible_connection=ssh ansible_user=ubuntu ansible_ssh_private_key_file=~/.ssh/${var.prefix} instance=${var.lightsail ? element(aws_lightsail_instance.myweb[*].name, 0) : element(aws_instance.myweb[*].tags["Name"], 0)}"
> filename = pathexpand("~/dev/ansible/hosts-aws")
> directory_permission = 0754
> file_permission = 0664
> }
> EOF
If you have gone through the AWS/Terraform against Lightsail article, then please delete the template file for the install script:
$ rm install.tf
Create the shell script for user_data:
$ cat << 'EOF' > scripts/install.sh
> #!/bin/bash
>
> MY_HOME="/home/ubuntu"
> export DEBIAN_FRONTEND=noninteractive
>
> # Install prereqs
> apt update
> apt install -y python3-pip apt-transport-https ca-certificates curl software-properties-common
> # Install docker
> curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
> add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
> apt update
> apt install -y docker-ce
> # Install docker-compose
> su ubuntu -c "mkdir -p $MY_HOME/.local/bin"
> su ubuntu -c "pip3 install docker-compose --upgrade --user && chmod 754 $MY_HOME/.local/bin/docker-compose"
> usermod -aG docker ubuntu
> # Add PATH
> printf "\nexport PATH=\$PATH:$MY_HOME/.local/bin\n" >> $MY_HOME/.bashrc
>
> exit 0
> EOF
Initialize the directory:
$ terraform init
Run a dry-run to see what will occur:
$ terraform plan -var 'lightsail=false'
Provision:
$ terraform apply -var 'lightsail=false' -auto-approve
Log on to the instance after a short while:
$ ssh -i ~/.ssh/myweb ubuntu@<The value of static_public_ip that was reported. One can also use 'terraform output static_public_ip' to print it again.>
Type yes and hit enter to accept.
On the host (a short while is needed for the run-once script to complete):
$ docker --version
$ docker-compose --version
$ logout
Tear down what was created by first performing a dry-run to see what will occur:
$ terraform plan -var 'lightsail=false' -destroy
Tear down the instance:
$ terraform destroy -var 'lightsail=false' -auto-approve
<–
References:
Source:
terraform_aws_myweb
