Making a Hugo Website The Full Stack Way pt 4 - CI/CD with Github Actions and Terraform

Making a Hugo Website The Full Stack Way pt 4 - CI/CD with Github Actions and Terraform

In the previous tutorial in our multi-part series on making a static website, we deployed a Google Cloud Storage bucket using Terraform. In this (very advanced) tutorial we will attempt to fully automate the deployment of our website by hooking it up with Github Actions for Continuous Integration/Continuous Deployment.

This tutorial is very optional and may be a bit overkill for folks who just want to make a website. For folks interested in simply making a website, it may be wise to stick to part 1 of this series

Goal

Our goal is to make it so that every time we push to Github, Github Actions will rebuild and re-deploy the Hugo Website we built in part 1. To do so we will need to setup Github Actions, add Terraform Modules to give Github Actions the proper permissions, and finally test out our deployment by pushing to Github.

Setup Github Actions

The first step is to setup Github Actions. To do this, copy the following into the file .github/workflows/github_actions.yaml

name: Blog CI/CD

on:
  workflow_dispatch:
  push:

jobs:
  build_deploy:
    name: Build and Deploy
    runs-on: ubuntu-20.04
    permissions:
      contents: 'read'
      id-token: 'write'
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3

      - name: Checkout submodules
        run: git submodule update --init --recursive
  
      - name: 'Authenticate to Google Cloud'
        id: 'auth'
        uses: 'google-github-actions/[email protected]'
        with:
          # Note: workload_identity_provider looks something like
          #   projects/123456789/locations/global/workloadIdentityPools/my-pool/providers/my-provider
          #   Look under "Default Audience" in the UI if setting this up there
          # Note: You must make sure to "Grant Access"/Connect Service account to your workload_identity_pool 
          workload_identity_provider: ${{ secrets.PROVIDER_NAME }} 
          service_account: ${{ secrets.SA_EMAIL }}

      - name: Setup Hugo
        env:
          HUGO_DOWNLOAD_URI: https://github.com/gohugoio/hugo/releases/download
          HUGO_VERSION: 0.101.0
          HUGO_FILE: hugo_extended_${HUGO_VERSION}_Linux-64bit.deb
        run: |
          curl -L ${HUGO_DOWNLOAD_URI}/v${HUGO_VERSION}/hugo_extended_${HUGO_VERSION}_Linux-64bit.deb --output ${HUGO_FILE}
          sudo dpkg -i ${HUGO_FILE}          

      - name: Deploy
        run: |
          hugo deploy --target=${{ secrets.DEPLOYMENT_TARGET }}

This YAML sets up Github actions to Authenticate with Google Cloud using a Workload Identity Provider and a Service Account.

Notice that we need to provide ${{ secrets.PROVIDER_NAME }}, ${{ secrets.SA_EMAIL }}, and {{ secrets.DEPLOYMENT_TARGET }}. You can simply replace {{ secrets.DEPLOYMENT_TARGET }} with the deployment target defined in your config.toml. However the other two require actual resources in Google Cloud.

Setting up a Workload Identity Provider and Service Account can be quite complicated, but its something we can automate with Terraform!

Adding Terraform modules

A Workload Identity Provider is a resource used to allow outside infrastructure (like Github Actions) to do things on your infrastucture. In this case, we need it for uploading files to a Cloud Storage Bucket.

The beauty of terraform is that you don’t have to write new modules for every little task. You can simply pull them from the internet! For example, a pre-written Open ID Connect module can be found here.

First, add the following to our main.tf

# google-beta Needed for OIDC module
provider "google-beta" {
  region = var.region
  credentials = file(var.key_file) # TODO: Replace with ADC: https://cloud.google.com/sdk/gcloud/reference/auth/application-default
  project = var.project_id
}

# The storage_service_account is used by github_oidc to update the blog
# It has full admin ability over storage for the project
# See: https://github.com/terraform-google-modules/terraform-google-service-accounts/blob/master/outputs.tf
module "storage_service_account" {
  source        = "terraform-google-modules/service-accounts/google"
  version       = "~> 3.0"
  project_id    = var.project_id
  prefix        = var.prefix
  names         = [var.service_account_name]
  display_name = "Storage admin service account"
  description = "Service account for managing storage access"
  project_roles = [
    "${var.project_id}=>roles/storage.objectAdmin", # Need to edit iam permissions for this service account
  ]
}


// https://github.com/terraform-google-modules/terraform-google-github-actions-runners/tree/master/modules/gh-oidc
module "oidc" {
  source = "terraform-google-modules/github-actions-runners/google//modules/gh-oidc"
  version = "3.1.0"
  project_id = var.project_id
  provider_id = var.oidc_provider_id
  pool_id = var.oidc_wif_pool_id
  issuer_uri = var.oidc_issuer_uri
  sa_mapping = {
    "storage-service-account" = {
      sa_name = "projects/${var.project_id}/serviceAccounts/${var.prefix}-${var.service_account_name}@${var.project_id}.iam.gserviceaccount.com"
      attribute = "*"
    }
  }
  depends_on = [
    module.storage_service_account
  ]
}

Also add the following to variables.tf

variable "prefix" {
    type = string
    default = "sa"
    description = "Service account prefix"
}

variable "oidc_wif_pool_id" {
    type = string
    description = "Pool ID for Open ID Connect Workload Identity Federation Pool"
}

variable "oidc_provider_id" {
    type = string
    description = "Open ID Connect provider id. ie: \"GitHub\""
    default = "GitHub"
}

variable "oidc_issuer_uri" {
    type = string
    description = "Open ID connect issuer ID"
    default = "https://token.actions.githubusercontent.com"
}

variable "service_account_name" {
    type = string
    default = "storage-service-account"
}

For outputs.tf we need two pieces of information: workload_provider_name and service_account_email, so lets add those to our outputs.tf

# Storage service account
output "storage_service_account_email" {
  description = "Storage Service account resource (for single use)."
  value       = module.storage_service_account.email
}

output "workload_provider_name" {
  value = module.oidc.provider_name
}
(Important) Pre-requisite - Updating your service account permissions

The service account used by our IaC tool (Terraform) will need increased permissions to do three tasks. One of these tasks is actually updating the capabilities of other service accounts! Here, rather than using our main service account, we are creating a new one (with more restricted permissions). In a large organization. This step can be done manually under IAM & Admin in the Google Cloud interface:

Allowing the service account used by Terraform to update IAM permissions

Adding Secrets to Github Actions

If you look at our github_actions.yaml you’ll notice the lines ${{ secrets.PROVIDER_NAME }} and ${{ secrets.SA_EMAIL }}. These secrets actually need to be added through Github under Settings >> Secrets.

You can lookup this information by looking for the workload_provider_name output of your Terraform state (terraform.tfstate) file if you’re storing this information locally.

Deploying

The steps from here should be fairly simply. Update terraform.tfvars with the new variables for the two modules we added, and run the following from prod/:

$ terraform plan -var-file terraform.tfvars -out terraform.tfplan
$ terraform apply -var-file terraform.tfvars

Conclusion

As you can see setting up automation can be quite an arduous task. If you’ve actually gotten this far, congratulations! You’ve not only put up a blog, but you now have the knowledge to deploy any resource on the cloud using some very modern DevOps practices!

Since you (presumably) used Terraform for setting up your static project, you can continue to build upon your infrastructure with things like:

Good luck!

Full Series

Example Template on Github

Related Posts

Why Big Tech Wants to Make AI Cost Nothing

Why Big Tech Wants to Make AI Cost Nothing

Earlier this week, Meta both open sourced and released the model weights for Llama 3.1, an extraordinarily powerful large language model (LLM) which is competitive with the best of what Open AI’s ChatGPT and Anthropic’s Claude can offer.

Read more
Host Your Own CoPilot

Host Your Own CoPilot

GitHub Co-pilot is a fantastic tool. However, it along with some of its other enterprise-grade alternatives such as SourceGraph Cody and Amazon Code Whisperer has a number of rather annoying downsides.

Read more
All the Activation Functions

All the Activation Functions

Recently, I embarked on an informal literature review of the advances in Deep Learning over the past 5 years, and one thing that struck me was the proliferation of activation functions over the past decade.

Read more