Building a Resume Website on AWS

Using Nginx, Terraform, Kubernetes, Docker, Jenkins and Sonarqube

My primary objective for this project is to demonstrate my expertise as a DevOps Engineer while expanding my technical skill set. I believe that true learning happens through hands-on experience and problem-solving.

Therefore, I will actively engage in the development process to showcase my knowledge and improve my abilities along the way.

Additionally, I aim for this project to serve as a comprehensive guide for individuals looking to establish their own DevOps infrastructure. By documenting the entire process and sharing my experiences, I hope to provide valuable insights and practical instructions that can assist others in successfully implementing similar systems.

Prerequisites

  1. AWS Account

    An AWS account is essential for using Terraform to build projects. It allows Terraform to interact with and manage AWS resources, facilitating infrastructure provisioning and management.

  2. Terraform Installed

    Terraform must be installed on your computer to build this project. Terraform is an infrastructure-as-code (IaC) tool that automates and manages cloud resources. Installing Terraform on your PC enables you to define your infrastructure as code and deploy it to AWS.

  3. Git Installed

    Git must be installed on your computer to clone the project from my GitHub repository. Git is a version control system that facilitates efficient collaboration and code change tracking.

Part 1: Infrastructure Setup

Deciding the Infrastructure Setup

For this stage of the project (highlighted in blue), I have chosen the following infrastructure setup. The image below illustrates the setup:

Infrastructure diagram showing AWS architecture

Instructions

Step 1: Choose a command terminal

Open a terminal (Command Prompt on Windows, Terminal on Mac or Linux).

Step 2: Choose a base folder

Navigate to the empty folder where you want to clone the repository:

cd /path/to/your/folder

Step 3: Clone the repository

Once you're in the right directory, use the git clone command followed by the URL of the repository:

git clone https://github.com/angpenghian/resume.git

Step 4: Navigate to the project folder

After cloning the GitHub repository, navigate to the terraform directory that includes the necessary Terraform and Bash scripts:

cd /Your-Folder-Directory/Resume-website/

Step 5: Create secrets folder

Create a new directory called 'secrets':

mkdir secrets

Step 6: Configure AWS credentials

Log into your AWS root user account. Use the console's search bar to find the IAM service.

AWS IAM search

Inside the IAM interface, navigate to the Users section and choose 'Add users'.

AWS IAM users

Follow the on-screen instructions to create your IAM user.

AWS IAM user creation

Once the user is created, click on it. In the 'Security credentials' tab, find 'Create access key'.

AWS IAM security credentials
AWS IAM create access key

It's crucial to save and document both keys for future reference.

AWS access keys

Step 7: Configure AWS key pairs

Create a key pair to enable SSH access into the AWS EC2 machines. Use the console's search bar to find "Key pairs".

AWS key pairs search

In the Key pairs section, select Create key pair. Take note of the region you're in because key pairs are region-specific.

AWS create key pair

Input your preferred key name.

AWS key pair name

Upon completion, the key pair will be generated and a .pem file will be downloaded.

AWS key pair created

Step 8: Place secrets and key pair

Save the access keys and key pairs in the given format within the 'secrets' directory.

Secrets folder structure

Step 9: Setup Terraform

Now that your security credentials are properly set up, there are still a few files you need to edit for the Terraform setup.

Terraform setup files

Step 10: Configure Terraform provider

In the provider file, you can modify the access key location if you've stored it elsewhere.

Terraform provider configuration

Step 11: Configure Terraform variables

For terraform.tfvars, you can modify parameters such as the EC2 machine image, your instance type, private key path, the domain you're using, and a validated SSL certificate ARN from AWS Cert Manager.

Terraform variables

Step 12: Initialize Terraform

Once you've finished editing the files, navigate to the terraform directory (Resume-website > terraform > server-based) and initialize:

terraform init
Terraform init output

Step 13: Apply Terraform

Navigate to the server-based directory and apply the infrastructure:

terraform apply
Terraform apply output

Step 14: Handle Terraform errors

After running terraform apply, it might have an error for remote-exec. Type terraform apply again.

terraform apply
Terraform apply error
Terraform apply error resolution

Step 15: Note the kubeadm join command

Once the initialization is complete, you will see a kubeadm join command. Take note of it and the IP addresses:

sudo kubeadm join [your-ip]:6443 --token [your-token] --discovery-token-ca-cert-hash [your-hash]
Kubeadm join command

Step 16: Change permission for secret key

Before SSH-ing into EC2, change the permission of the key.pem file:

chmod 400 [your-key.pem]
chmod 400 key permission

Step 17: SSH into node01

Access the server via SSH to setup the Nginx Kubernetes deployment on node01:

ssh -i [your-key.pem] ec2-user@[your-ip]
SSH into node01

Step 18: Join node01 to the Kubernetes cluster

Using the previous kubeadm join command, join node01 to the cluster.

Node01 kubeadm join

Step 19: Check node status

SSH into the master node and check the status of all nodes:

ssh -i [your-key.pem] ec2-user@[your-ip]
kubectl get node
Master node SSH and kubectl get node

Step 20: Navigate to Kubernetes YAML files

Navigate to the nginx directory on the master node:

cd /home/ec2-user/website/kubernetes-setup
Kubernetes setup directory

Step 21: How the website files got to the server

The website folder is copied to the EC2 instance via the Terraform provisioner in 6-ec2.tf:

Terraform provisioner configuration

Step 22: Create Nginx deployment

Deploy the nginx deployment on the Kubernetes cluster:

kubectl apply -f kubernetes-nginx-setup.yaml
kubectl apply nginx

Step 23: Check deployment status

kubectl get all
kubectl get all

Step 24: Verify the website

Check if the nginx deployment is working by opening the URL in a browser.

Website after nginx deployment

Step 25: Domain name configuration

The domain is configured in the Terraform file 8-route53.tf, which creates a record set for angpenghian.com:

Route53 A record configuration
Part 1 Complete! The infrastructure setup with Nginx on Kubernetes is done. Part 2 covers CI/CD with Jenkins.

Part 2: CI/CD Pipeline Setup

Now that we have completed the infrastructure setup, we can set up the CI/CD pipeline. For this stage (highlighted in red):

Infrastructure diagram with CI/CD highlighted

Step 1: SSH into Jenkins

Navigate to the secrets directory and SSH into the Jenkins EC2 instance:

ssh -i [your-key.pem] ec2-user@[your-ip]
SSH into Jenkins

Step 2: Join Jenkins to the Kubernetes cluster

sudo kubeadm join [your-ip]:6443 --token [your-token] --discovery-token-ca-cert-hash [your-hash]
Jenkins kubeadm join

Step 3: Verify Jenkins node status

SSH into the master node and check node status:

ssh -i [your-key.pem] ec2-user@[your-ip]
kubectl get node
Master node with Jenkins

Step 4: Navigate to Kubernetes YAML files

cd /home/ec2-user/website/kubernetes-setup
Kubernetes setup directory

Step 5: Create Jenkins deployment

kubectl apply -f kubernetes-jenkins-setup.yaml
kubectl apply Jenkins

Step 6: Verify Jenkins website

Check if the Jenkins deployment is working by opening it in a browser.

Jenkins website

Step 7: Jenkins domain name

The Jenkins domain is configured in 8-route53.tf with a record set for jenkins.angpenghian.com:

Route53 Jenkins A record

Step 8: Get Jenkins initial admin password

SSH into the master node and get the Jenkins pod name:

kubectl get pods --namespace=devops-tools
Get Jenkins pod name

View the Jenkins pod log to find the initial admin password:

kubectl logs [Jenkins-pod-name] --namespace=devops-tools
Jenkins initial admin password

Step 9: Unlock Jenkins

Enter the initial admin password and click continue.

Unlock Jenkins

Step 10: Install suggested plugins

Click on install suggested plugins and wait for the installation to complete.

Jenkins install plugins
Jenkins plugins installing

Step 11: Create first admin user

Input your username, password, full name and email. Save and continue.

Jenkins first admin user

Step 12: Instance configuration

Click on save and finish.

Jenkins instance configuration
Start using Jenkins

Step 13: Create a new pipeline

Click on new item and select freestyle project. Name it (e.g., "resume-website") and click OK.

Jenkins new item
Jenkins freestyle project

Step 14: Source code management

Select Git and input the repository URL:

https://github.com/angpenghian/Resume-website

Select the branch (main).

Jenkins source code management

Step 15: Build triggers

Select "GitHub hook trigger for GITScm polling" and save.

Jenkins build triggers

Step 16: Configure GitHub webhooks

In your GitHub repository settings, navigate to Webhooks.

GitHub settings
GitHub webhooks

Click "Add webhook" and enter the payload URL:

GitHub add webhook
GitHub payload URL

Select "Let me select individual events" and check Push and Pull Request.

GitHub select events
GitHub push and pull request events

Step 17: Test the build

Go back to Jenkins and click "Build Now".

Jenkins build now
Jenkins build successful

Step 18: Configure build steps

Click configure, select Build Steps, and add an Execute Shell step with:

scp -r /var/jenkins_home/workspace/resume-website/website/* ec2-user@<masternode-public-ip>:/home/ec2-user/website/
Jenkins configure
Jenkins build steps
Jenkins execute shell

Step 19: Set passwords for EC2 instances

SSH into the master node and node01, then set passwords:

sudo passwd ec2-user
Set password on masternode
Set password on node01

Step 20: Setup SSH keys for Jenkins

Get the Jenkins pod name and SSH into it:

kubectl get pods --namespace=devops-tools
Get Jenkins pod name
kubectl exec -it [jenkins-pod-name] --namespace=devops-tools -- /bin/bash
Exec into Jenkins pod

Generate SSH keys inside the Jenkins pod:

ssh-keygen
Jenkins SSH keygen

Copy the public key to the EC2 instances:

ssh-copy-id ec2-user@masternode-public-ip
Jenkins SSH copy ID

Step 21: Test the CI/CD pipeline

The live website is still on the old version:

Old website version

Make a change to the website and push to GitHub:

HTML changes
git add .
git commit -m 'update'
git push
Git push

Jenkins picks up the change and builds automatically:

Jenkins build pending
Jenkins build done

The updated website is now live:

New website version
CI/CD Pipeline Complete! The full infrastructure with automated deployments via Jenkins webhooks is now operational.