Long time no see! What’s new, you ask?

Feels funny posting more than a year after my last post, starting with the same sentence. 😅

It is what it is though. I’ve been thinking about blogging but just haven’t made the time. So very belatedly, here’s the latest.

• The new job is no longer new. I’m primarily working with Google Cloud Platform (GCP) and GitLab.
• The new house is no longer new. I’m so happy to be able to live where I do!
• I have picked up two professional certifications in 2023 so far, the GCP Cloud Digital Leader and Cloud Architect.
• So much learning about 3D printers, printing and CAD. I now own four printers. D:
• I had an international holiday. It was a bit of a disaster, quite literally. I ended up stuck in a hotel while a tropical cyclone passed.
• I went to like, 5 weddings. Love is in the air!

I’m still learning new things about just about everything, all the time. My life feels very busy and yet when I look back at the days, weeks and months past, less has been achieved than I hoped. I wonder how people with kids get anything done, haha! My sleep is bad enough already. D:

Well, a few things;

• Bought and moved into a house with my partner,
• Built gadgets to improve life in said house,
• Finally built the RigidBot Big 3D printer that has been sitting unassembled at my parents house for some 6-7 years, 😬
• Started a new job.

On a completely unrelated note and in no particular order, I’ve started learning a lot about;

• house maintenance and renovation,
• micro-controllers and electronics in general,
• 3D printer construction/assembly, upgrading and troubleshooting, 😓
• Google Cloud Platform.

To say it has been busy is somewhat of an understatement. Everything has a steep learning curve, which is tiring but rewarding. The awkward part is knowing that I have still barely scratched the surface.

Time and persistence will pay off, but hard slog is still there.

I’m planning to write up some of the projects as blog posts, so keep an eye out for those sometime this year. 😅

In my defence, it has been very busy. I’ll be posting about that soon.

Here are some of the extra bibs and bobs that I put on my DOKS 2021 Challenge cluster.

Some extra tools are needed here, or at least, I make use of them.

• kubectl, ‘lets you control Kubernetes clusters’
• doctl, optional ‘the official CLI tool for interacting with DO resources’
• k9s, optional a helpful TUI for interacting with k8s resources

You can download the kubeconfig for the cluster from the DO gui, doctl or extract it from Terraform state. I’ll be using doctl just because I wanted to try it.

$ brew install doctl
==> Downloading https://ghcr.io/v2/homebrew/core/doctl/manifests/1.67.0
######################################################################## 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/doctl/blobs/sha256:7c235dbbf49483325e38b76cb72263ec9301e1a6ffd958ee5d15
==> Downloading from https://pkg-containers.githubusercontent.com/ghcr1/blobs/sha256:7c235dbbf49483325e38b76cb72263ec930
######################################################################## 100.0%
==> Pouring doctl--1.67.0.x86_64_linux.bottle.tar.gz
==> Caveats
zsh completions have been installed to:
  /home/linuxbrew/.linuxbrew/share/zsh/site-functions
==> Summary
🍺  /home/linuxbrew/.linuxbrew/Cellar/doctl/1.67.0: 9 files, 24.6MB

$ doctl auth init
Please authenticate doctl for use with your DigitalOcean account. You can generate a token in the control panel at https://cloud.digitalocean.com/account/api/tokens

Enter your access token:
Validating token... OK

$ doctl kubernetes cluster kubeconfig save 898db296-575e-4112-a651-4d0c429c5b28
Notice: Adding cluster credentials to kubeconfig file found in "/home/liam/.kube/config"
Notice: Setting current-context to do-sgp1-do-k8s-2021

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.4", GitCommit:"e87da0bd6e03ec3fea7933c4b5263d151aafd07c", GitTreeState:"clean", BuildDate:"2021-02-18T16:12:00Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.5", GitCommit:"aea7bbadd2fc0cd689de94a54e5b7b758869d691", GitTreeState:"clean", BuildDate:"2021-09-15T21:04:16Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/amd64"}

Okay, good to go.

Loki / Grafana stack

Update main.tf and apply.

resource "helm_release" "loki" {
  name       = "loki"
  repository = "https://grafana.github.io/helm-charts"
  chart      = "loki-stack"
  version    = "2.5.0"

  namespace        = "loki"
  create_namespace = true

  set {
    name  = "grafana.enabled"
    value = "true"
  }

  set {
    name  = "prometheus.enabled"
    value = "true"
  }
}

Get the admin password for the initial Grafana login, and port forward so that you have access to Grafana locally.

$ kubectl get secret --namespace loki loki-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
<password output here>

$ kubectl port-forward --namespace loki service/loki-grafana 8080:80
Forwarding from 127.0.0.1:8080 -> 3000
Forwarding from [::1]:8080 -> 3000

Hit http://localhost:8080 and you should be staring at a Grafana login page. Success! Deep diving on Grafana is a bit much for this post, but have a play around in the explore tab on the left after you login. You’ll already have logs (thanks to Loki) and cluster metrics (thanks to Prometheus) available to view.

One fun thing I did was check what output there was in Falco when I opened up a shell on my Loki pod. I used the LogQL search term {job="falco/falco"}, i.e. give me all the logs for the falco job in the falco namespace, hit search and voila!

2021-11-27T07:38:52.832748533Z stdout F 07:38:52.830148939: Notice A shell was spawned in a container with an attached terminal (user=<NA> user_loginuid=-1 k8s.ns=loki k8s.pod=loki-0 container=269597121f05 shell=sh parent=<NA> cmdline=sh terminal=34816 container_id=269597121f05 image=docker.io/grafana/loki) k8s.ns=loki k8s.pod=loki-0 container=269597121f05
2021-11-27T07:38:52.832551682Z stdout F 07:38:52.830014807: Notice A shell was spawned in a container with an attached terminal (user=<NA> user_loginuid=-1 k8s.ns=loki k8s.pod=loki-0 container=269597121f05 shell=sh parent=<NA> cmdline=sh -c command -v bash >/dev/null && exec bash || exec sh terminal=34816 container_id=269597121f05 image=docker.io/grafana/loki) k8s.ns=loki k8s.pod=loki-0 container=269597121f05

Maybe something like count_over_time({job="falco/falco"} |= "A shell was spawned in a container with an attached terminal"[1m]) will be a useful one to keep an eye on. 🤔

Alrighty, time to add some Drone CI to the mix to get those automatic releases happening. First things first, let’s start Drone CI and get it connected to Gitea. Drone CI provide reasonable documentation, so I won’t go into all the details but basically:

1. Create an OAuth app in Gitea.
   • Make note of the Client Secret when shown to you, as it’s the only time it’ll be shown.
   • Make sure you provide the appropriate Redirect URI. In our case, this should be http://drone:8080/login.
2. Create a secret, however you want really. We’ll use this for the `
3. Add the below to our existing docker-compose file.
   • Be sure to provide the appropriate Client ID, Client Secret and DRONE_RPC_SECRET. Make something up for that last one, it can be whatever you want. Ideally something strong. 😄

  drone:
    image: drone/drone:2
    container_name: drone
    environment:
      - DRONE_SERVER_HOST=drone:8080
      - DRONE_SERVER_PROTO=http
      - DRONE_GITEA_CLIENT_ID=<your-client-id-here>
      - DRONE_GITEA_CLIENT_SECRET=<your-client-secret-here>
      - DRONE_GITEA_SERVER=http://gitea:3000
      - DRONE_RPC_SECRET=<your-rpc-secret-here>
      - DRONE_USER_CREATE=username:gitea,admin:true
    volumes:
      - drone:/data
    ports:
      - 8080:80
    restart: unless-stopped

  runner:
    image: drone/drone-runner-docker:1
    container_name: runner
    environment:
      - DRONE_RPC_PROTO=http
      - DRONE_RPC_HOST=drone
      - DRONE_RPC_SECRET=<your-rpc-secret-here>
      - DRONE_RUNNER_CAPACITY=2
      - DRONE_RUNNER_NAME=jovial-easley
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    depends_on:
      - drone
    restart: unless-stopped

Let’s start Drone CI now.

$ docker-compose up -d
Recreating gitea ... done
Creating drone   ... done
Creating runner  ... done

Hit http://drone:8080 and you should see Hello, Welcome to Drone.. Awesome! Click continue and you should be redirected to Gitea with a prompt to authorize drone to access your account. Accept and you should be redirected back to Drone to Complete your Drone Registration.. I actually just leave these blank and hit submit, it’s fine.

Sweet, now I can see gitea/crimp-city listed under repositories. Click on it and then on Activate Repository. When you do this, Drone CI creates a webhook in Gitea, telling it to send it an update when a commit is made. You can ignore everything else for now.

So Gitea knows it should tell Drone CI when this repo gets a new commit. Let’s test this out. Create a .drone.yml file in the root of the crimp-city repo.

# .drone.yml

kind: pipeline
name: default

steps:
  - name: run
    image: golang
    commands:
      - go run main.go

Add, commit and push this change up to Gitea.

$ git add .drone.yml
$ git commit -m "Add drone ci config"
[master 693d656] Add drone ci config
 1 file changed, 10 insertions(+)
 create mode 100644 .drone.yml
$ git push
Enumerating objects: 8, done.
Counting objects: 100% (8/8), done.
Delta compression using up to 12 threads
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 686 bytes | 686.00 KiB/s, done.
Total 6 (delta 2), reused 0 (delta 0), pack-reused 0
remote: . Processing 1 references
remote: Processed 1 references in total
To http://gitea:3000/gitea/crimp-city.git
   4836926..693d656  master -> master

I checked the build tab in Drone and… nothing. Huh, that’s odd. I venture over to the crimp-city repo in Gitea, open the settings and webhooks tab, scrolling down to the bottom and, ah. A failed delivery.

Delivery: Post "http://drone:8080/hook?secret=kzpOpiciNP5TJDv2x3nKh4lb3MzNiI3n": dial tcp 172.23.0.3:8080: connect: connection refused

Oh. Yes. More network fun. Gitea is using the internal docker-compose network, where Drone is actually listening on port 80 and not 8080. We can fix this easily by scrolling back up the page and updated the Target URL to http://drone/hook?secret=kzpOpiciNP5TJDv2x3nKh4lb3MzNiI3n.

To test this is now working, we can add a README to crimp-city.

# crimp-city

Testing that Gitea can talk to Drone CI!

Do the git add/commit/push dance and then check that Drone CI builds tab again. Sweet! Success. I can click through and view output from the two stages of the build, the clone and the run step.

Time to make it actually do releases though. Update the Drone CI config file.

# .drone.yml

kind: pipeline
name: default

steps:
  - name: fetch
    image: docker:git
    commands:
      - git fetch --tags

  # we don't have tests, awkward
  # - name: test
  #   image: golang:1.17
  #   volumes:
  #     - name: deps
  #       path: /go
  #   commands:
  #     - go test -race -v ./... -cover

  - name: release
    image: golang:1.17
    environment:
      GITEA_TOKEN:
        from_secret: gitea_token
    volumes:
      - name: deps
        path: /go
    commands:
      - curl -sL https://git.io/goreleaser | bash
    when:
      event:
        - tag

volumes:
  - name: deps
    temp: {}

Why do we have the fetch step? Per the GoRelease Drone CI documentation:

By default, drone does not fetch tags. plugins/git is used with default values, in most cases we’ll need overwrite the clone step enabling tags in order to make goreleaser work correctly.

One thing we’ll need to do is set a secret in Drone CI, so that GoReleaser can talk to Gitea. You may remember this as our GITEA_TOKEN. You can use the same one as before or create a new dedicated secret. If it was production, I’d do the latter.

To set the secret, go into Drone CI, click through to the crimp-city repo, Settings -> Secrets -> + New Secret. Name it gitea_token and the value to the secret, then create.

Git add/commit/push the updated Drone CI config. Time to check what happened in the latest build logs from the runner. We have our clone and fetch steps but no release? This is because set to only run that step when the commit has a tag. Let’s try that out now.

$ git tag -a v0.1.1 -m "Testing Drone CI and GoReleaser"
$ git push origin v0.1.1
Enumerating objects: 1, done.
Counting objects: 100% (1/1), done.
Writing objects: 100% (1/1), 180 bytes | 180.00 KiB/s, done.
Total 1 (delta 0), reused 0 (delta 0), pack-reused 0
remote: . Processing 1 references
remote: Processed 1 references in total
To http://gitea:3000/gitea/crimp-city.git
 * [new tag]         v0.1.1 -> v0.1.1

Check out the builds again annnnnnnnd…

It ran through the release stage as well! And if we check Gitea, we can see that there are now two releases. Awesome!

Here we conclude. I actually finished writing a thing. Word. I’m considering creating video tutorials and putting them up on YouTube because man, is it effort writing all this out. That’s probably my bad for not tactically using screenshots though, so, I apologise. Get in touch on Twitter or elsewhere if you have any questions or difficulties.

Peace out and I’ll see you in my next post.

So as you may already know or have guessed, this blog is built using Hugo, and is hosted on GitLab! There’s documentation on this already, so I won’t rehash that². No, I’m here to talk about posting, in the future.

When you’re writing a post in Hugo, you can set the post date to a time in the future, but unless you have the buildFuture configuration set, the Hugo won’t build that post.

Why would you want to set a date in the future? Maybe you know you’ll be away and unable to post for a few weeks, or you’re just feeling productive and write a heap of content. Instead of overwhelming your readers with a tower of posts released on the same day, you could stagger that content over several weeks.

But here’s the thing. With the default configuration following the documentation linked earlier, the blog only builds when the pipeline runs, which means you making a new commit to the main branch.

When I’m away, I’m away! 😎 🌴 I can’t be triggering these builds!

Happily, GitLab lets you set timer. They’re called pipeline schedules, and you can either set one with the ever powerful cron syntax or one of their default options. I’m using 0 20 * * * in the UTC timezone, meaning about 4am local.

So I can just write, set the date I want and make sure to remove the draft status and… fresh posts for the people when they rise and shine! ☕

Howdy again, hope you’re all going well. Let’s get right to it.

GoReleaser is already installed in the devcontainer environment for you, let’s check that it’s happy.

$ pwd
/workspaces/go-goreleaser-drone-example/crimp-city
$ goreleaser -h
GoReleaser is a release automation tool for Go projects.
Its goal is to simplify the build, release and publish steps while providing
variant customization options for all steps.

GoReleaser is built for CI tools, you only need to download and execute it
in your build script. Of course, you can also install it locally if you wish.

You can also customize your entire release process through a
single .goreleaser.yml file.

Usage:
  goreleaser [command]

Available Commands:
  build       Builds the current project
  check       Checks if configuration is valid
  completion  generate the autocompletion script for the specified shell
  help        Help about any command
  init        Generates a .goreleaser.yml file
  jsonschema  outputs goreleaser's JSON schema
  release     Releases the current project

Flags:
      --debug     Enable debug mode
  -h, --help      help for goreleaser
  -v, --version   version for goreleaser

Use "goreleaser [command] --help" for more information about a command.

Excellent, everything seems to be in order here. Let’s generate the default config.

$ goreleaser init
   • Generating .goreleaser.yml file
   • config created; please edit accordingly to your needs file=.goreleaser.yml

It should look something like the below, though I’ve made some small changes to make it only build for 64-bit linux hosts.

before:
  hooks:
    - go mod tidy
builds:
  - env:
      - CGO_ENABLED=0
    goos:
      - linux
    goarch:
      - amd64
archives:
  - replacements:
      linux: Linux
      amd64: x86_64
checksum:
  name_template: 'checksums.txt'
snapshot:
  name_template: "{{ incpatch .Version }}-next"
changelog:
  sort: asc
  filters:
    exclude:
      - '^docs:'
      - '^test:'

As usual, let’s test this works before moving on.

$ goreleaser release --snapshot --rm-dist
   • releasing...
   • loading config file       file=.goreleaser.yml
   • loading environment variables
   • getting and validating git state
      • ignoring errors because this is a snapshot error=git doesn't contain any tags. Either add a tag or use --snapshot
      • building...               commit=9dac672bfae18f00162a9c96a6bc87197c652d4b latest tag=v0.0.0
      • pipe skipped              error=disabled during snapshot mode
   • parsing tag
   • running before hooks
      • running                   hook=go mod tidy
   • setting defaults
      • snapshotting
      • github/gitlab/gitea releases
      • project name
      • loading go mod information
      • building binaries
      • universal binaries
      • creating source archive
      • archives
      • linux packages
      • snapcraft packages
      • calculating checksums
      • signing artifacts
      • signing docker images
      • docker images
      • docker manifests
      • artifactory
      • blobs
      • homebrew tap formula
      • gofish fish food cookbook
      • scoop manifests
      • discord
      • reddit
      • slack
      • teams
      • twitter
      • smtp
      • mattermost
      • milestones
      • telegram
   • snapshotting
      • building snapshot...      version=0.0.1-next
   • checking ./dist
      • --rm-dist is set, cleaning it up
   • loading go mod information
   • writing effective config file
      • writing                   config=dist/config.yaml
   • building binaries
      • building                  binary=/workspaces/go-goreleaser-drone-example/crimp-city/dist/crimp-city_linux_amd64/crimp-city
   • archives
      • creating                  archive=dist/crimp-city_0.0.1-next_Linux_x86_64.tar.gz
   • calculating checksums
      • checksumming              file=crimp-city_0.0.1-next_Linux_x86_64.tar.gz
   • release succeeded after 0.29s

Okay, good, and what about the binary that it actually produced?

$ ./dist/crimp-city_linux_amd64/crimp-city

█░█ █▀▀ █░░ █░░ █▀█   █░█░█ █▀█ █▀█ █░░ █▀▄
█▀█ ██▄ █▄▄ █▄▄ █▄█   ▀▄▀▄▀ █▄█ █▀▄ █▄▄ █▄▀ version 0.0.1-next

Nice! But what’s this? The version has changed from dev to 0.0.1-next. Why? Well, Go lets you pass in variables at build time, and GoReleaser has some defaults that it will pass in.

# Custom ldflags templates.
# Default is `-s -w -X main.version={{.Version}} -X main.commit={{.Commit}} -X main.date={{.Date}} -X main.builtBy=goreleaser`.

Normally GoReleaser will do some magic with the most recent git tag for the version, but we don’t currently have a tag, so it supplies 0.0.1-next for us.

So, we have a Go program and a local working GoReleaser config. Let’s make things official and actually make a release in Gitea. We’ll need to create an API token for GoReleaser to use, and tell GoReleaser how to reach Gitea.

GoReleaser requires an API token to deploy the artifacts to Gitea. You can create one in Settings | Applications | Generate New Token page of your Gitea instance.

This token should be added to the environment variables as GITEA_TOKEN.

Set the token you just created.

export GITEA_TOKEN=<your-token-here>

Update the .goreleaser.yml configuration.

Now, I think there’s a bug in the current version of GoReleaser. I’ve raised an issue for it, but basically, the release block included below shouldn’t be necessary. If you’re following along, try without it and if you get a 404 error, add it in.

I found a bug in GoReleaser, and it was fixed almost immediately! Awesome work. If you’re on v1.1.0 of GoReleaser, you can leave the commented part of the block below as is. If you’re on a lower version, you need to uncomment it. Or just upgrade. 🤷

gitea_urls:
  api: http://gitea:3000/api/v1/
  download: http://gitea:3000
# release:
#   gitea:
#     owner: gitea
#     name: crimp-city

Now commit the changes, tag the commit and push it to Gitea.

$ git add .goreleaser.yml
$ git commit -m "Update gitea goreleaser config"
[master 4836926] Update gitea goreleaser config
 1 file changed, 4 insertions(+)
$ git tag -a v0.1 -m "First release"
$ git push origin v0.1
Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
remote: . Processing 1 references
remote: Processed 1 references in total
To http://gitea:3000/gitea/crimp-city.git
 * [new tag]         v0.1 -> v0.1
$ goreleaser release --rm-dist
   • releasing...
   • loading config file       file=.goreleaser.yml
   • loading environment variables
   • getting and validating git state
      • building...               commit=4836926dc37b8e8cac913f7f0e6f8ae5d2376c06 latest tag=v0.1
   • parsing tag
   • running before hooks
      • running                   hook=go mod tidy
   • setting defaults
      • snapshotting
      • scm releases
      • project name
      • loading go mod information
      • building binaries
      • universal binaries
      • creating source archive
      • archives
      • linux packages
      • snapcraft packages
      • calculating checksums
      • signing artifacts
      • signing docker images
      • docker images
      • docker manifests
      • artifactory
      • blobs
      • homebrew tap formula
      • krew plugin manifest
      • gofish fish food cookbook
      • scoop manifests
      • discord
      • reddit
      • slack
      • teams
      • twitter
      • smtp
      • mattermost
      • milestones
      • linkedin
      • telegram
   • checking ./dist
      • --rm-dist is set, cleaning it up
   • loading go mod information
   • writing effective config file
      • writing                   config=dist/config.yaml
   • generating changelog
      • writing                   changelog=dist/CHANGELOG.md
   • building binaries
      • building                  binary=/workspaces/go-goreleaser-drone-example/crimp-city/dist/crimp-city_linux_amd64/crimp-city
   • archives
      • creating                  archive=dist/crimp-city_0.1_Linux_x86_64.tar.gz
   • calculating checksums
      • checksumming              file=crimp-city_0.1_Linux_x86_64.tar.gz
   • publishing
      • scm releases
         • creating or updating release repo= tag=v0.1
         • Gitea release created     id=3
         • uploading to release      file=dist/checksums.txt name=checksums.txt
         • uploading to release      file=dist/crimp-city_0.1_Linux_x86_64.tar.gz name=crimp-city_0.1_Linux_x86_64.tar.gz
   • announcing
   • release succeeded after 1.44s

Great success. If you go browse to the releases page for your repo¹, you’ll see a new release, including:

• a changelog of all the commits
• zipped archives of the source code
• a zipped copy of the crimp-city binary
• a checksums.txt so that people can confirm their copy of the downloaded binary is legitimate.

How good is that? And you remember how we had the version embedded in the binary output? Yep, that’s using the git tag we set.

$ ./dist/crimp-city_linux_amd64/crimp-city

█░█ █▀▀ █░░ █░░ █▀█   █░█░█ █▀█ █▀█ █░░ █▀▄
█▀█ ██▄ █▄▄ █▄▄ █▄█   ▀▄▀▄▀ █▄█ █▀▄ █▄▄ █▄▀ version 0.1

Nice. Alright, that’s all for now. Next time we’ll get Drone CI to automatically create a release for us when we push a tag to Gitea.

In this post, we’ll create our Gitea container, a git repo to hold our code, and a very basic hello world program in Go.

I’m going to assume you already have Docker Desktop¹, vscode and Remote - Containers extension installed. If you don’t, go do that now. I also assume some basic knowledge about git and how to use git repository hosting services like Github.

I’ve provided a vscode devcontainer configuration which includes the tools you’ll need for the entire project, so to make things easy I recommend you use that.

Start by cloning my example repository and then checkout 10d412a, this commit includes only the necessary devcontainer and docker-compose for bringing up Gitea. Finally, open the folder in vscode.

git clone https://github.com/iJebus/go-goreleaser-drone-example.git
cd go-goreleaser-drone-example
git checkout 10d412a
code .

You should be prompted by vscode to reopen in a devcontainer, which you should do. The devcontainer will then build and finally you’ll be ready to go. Time to bring up Gitea.

# use the terminal on the docker host or inside the vscode devcontainer, it's the same one either way
$ docker-compose up -d
Starting gitea ... done

If you followed my preferred method from the previous post and have updated your /etc/hosts file, you should now access Gitea at http://gitea:3000. You’ll see a lot of configuration which you can ignore, just scroll to the bottom and hit Install Gitea. There will be a brief loading screen, after which you should click Register in the top right. I recommend the username gitea, and later steps will assume this, but all other values can be set however you like.

Create a new project in Gitea. This is the project that’s going to host our Golang project, including the GoReleaser and Drone CI configuration. I’m going bouldering later today so I’ll call mine crimp-city. Clone this project somewhere in the devcontainer², as the devcontainer already has the current latest version of Go installed.

$ pwd
/workspaces/go-goreleaser-drone-example
$ git clone http://gitea:3000/gitea/crimp-city.git
Cloning into 'crimp-city'...
warning: You appear to have cloned an empty repository.

Now to create our basic go program.

$ cd crimp-city
$ go mod init crimp-city
go: creating new go.mod: module crimp-city
$ touch main.go

Open main.go in vscode and fill it with something like the below.

package main

import "fmt"

var version = "dev"

func main() {
	fmt.Printf(`
█░█ █▀▀ █░░ █░░ █▀█   █░█░█ █▀█ █▀█ █░░ █▀▄
█▀█ ██▄ █▄▄ █▄▄ █▄█   ▀▄▀▄▀ █▄█ █▀▄ █▄▄ █▄▀ version %s
`, version)
}

Let’s test that works.

$ go run main.go

█░█ █▀▀ █░░ █░░ █▀█   █░█░█ █▀█ █▀█ █░░ █▀▄
█▀█ ██▄ █▄▄ █▄▄ █▄█   ▀▄▀▄▀ █▄█ █▀▄ █▄▄ █▄▀ version dev

Sweet! Let’s commit these back to Gitea and call it a day!

$ git add go.mod main.go
$ git commit -m "initial commit"
[master (root-commit) 9dac672] initial commit
 2 files changed, 15 insertions(+)
 create mode 100644 go.mod
 create mode 100644 main.go
$ git push
Enumerating objects: 4, done.
Counting objects: 100% (4/4), done.
Delta compression using up to 12 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (4/4), 424 bytes | 424.00 KiB/s, done.
Total 4 (delta 0), reused 0 (delta 0), pack-reused 0
remote: . Processing 1 references
remote: Processed 1 references in total
To http://gitea:3000/gitea/crimp-city.git
 * [new branch]      master -> master

Goovy, see you in the next post where we’ll be implementing GoReleaser.

First steps

I created a repository on GitHub to hold the relevant code, if you go look now you should see the completed project. Noice.

Accepting the challenge and signing up for a DigitalOcean (DO) account was pretty simple. There’s a little asterisk on the billing dashboard mentioning that it’s updated daily, so don’t be surprised when applying coupons and not seeing the Remaining account credits field update. Despite the warning, I found this a little disappointing. I got the message saying the credits were applied, so, why can’t the dashboard reflect that?

As is pretty standard for me these days, one of the first things I did was add a vscode devcontainer configuration so that I have a nice portable contained¹ environment for working in. I can’t tell you how nice I find being able to move from one computer to another, pull the repo and basically be ready to go. That said, it’s actually a bit overkill for this project because I only actually needed the terraform binary. You’ll see why soon.

Finally, I took a bit of a peek at the TF documentation for the DO provider. I saw the provider needs an API token to be able to interact with DO on my behalf, which makes total sense, so I created a Personal Access Token called do-k8s-2021 from the API page. I’m only using the token with this project, so I can confidently delete it in future and know what will be impacted. Anyway, I copied down the token temporarily as it’s only shown once.

To make things a little more interesting, I decided to use Terraform Cloud (TFC) for the first time². This involved creating a new workspace named do-k8s-2021 to keep things consistent, and selected the CLI-driven workflow option. I was provided with instructions and sample TF code to get started, and everything just worked³. You might be asking why, at this point.

Well, I’m glad you asked.

1. It’s something that I wanted to learn more about and get practical experience using.
2. It makes development even more portable.
   • Sure, I could store the terraform state in the repo using the local backend, but this is a public project so anyone could see it⁴, and I feel like it’s a bad practice anyway. The solution? Use a remote state backend.
   • Generally I’m using AWS, so I use S3 and DynamoDB for state and state-locking respectively, but this project isn’t on AWS. It’s on DigitalOcean. What to do? 🤔
   • Terraform Cloud provides remote state for free. Even better, I can securely store the DO token here as well. That means I don’t have to worry about how to distribute the token between development machines. Noice.

DigitalOcean

So, what did I have at this point?

• A DO account
• A TFC account and workspace, configured with my DO Personal Access Token exported as a sensitive Workspace variable called DIGITALOCEAN_TOKEN
• A nearly empty main.tf

First things first, I needed to set up the DO provider in TF so I updated the main.tf.

# main.tf
terraform {
  backend "remote" {
    organization = "ljones"

    workspaces {
      name = "do-k8s-2021"
    }
  }
  required_providers {
    digitalocean = {
      source = "digitalocean/digitalocean"
      version = "~> 2.16.0"
    }
  }
}

provider "digitalocean" {}

The ~> means that the DO provider can update to any 2.16.x version. Side node, it’s important to run terraform init after updating providers so that new providers are downloaded (if running locally) and the .terraform.lock.hcl file is created. The TL;DR is that this file keeps helps keep things consistent, and it should be committed to version control with the rest of the code.

Before I went slapping more TF code in there, I had a look around the DO console. Featured quite prominently was first-project. Huh, indeed. What’s that?

Projects let you organize your DigitalOcean resources (like Droplets, Spaces, and load balancers) into groups that fit the way you work. You can create projects that align with the applications, environments, and clients that you host on DigitalOcean.

Yep, cool, I definitely want one of those to house all the resources associated with this, uh, project. Back to the documentation again and lo, there is indeed a resource for managing DO projects. I added the below to my main.tf and ran a plan operation.

Note, I usually have terraform aliased to tf for convenience, so don’t be surprised when you see tf being used in the terminal!

resource "digitalocean_project" "do-k8s-2021" {
  name        = "do-k8s-2021"
  description = "DigitalOcean Kubernetes Challenge 2021"
  purpose     = "Deploy a security and compliance system"
}

$ tf plan
Running plan in the remote backend. Output will stream here. Pressing Ctrl-C
will stop streaming the logs, but will not stop the plan running remotely.

Preparing the remote plan...

To view this run in a browser, visit:
https://app.terraform.io/app/ljones/do-k8s-2021/runs/run-bVsGoY2hdCxXkpDY

Waiting for the plan to start...

Terraform v1.0.11
on linux_amd64
Configuring remote state backend...
Initializing Terraform configuration...

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # digitalocean_project.do-k8s-2021 will be created
  + resource "digitalocean_project" "do-k8s-2021" {
      + created_at  = (known after apply)
      + description = "DigitalOcean Kubernetes Challenge 2021"
      + environment = "Development"
      + id          = (known after apply)
      + is_default  = (known after apply)
      + name        = "do-k8s-2021"
      + owner_id    = (known after apply)
      + owner_uuid  = (known after apply)
      + purpose     = "Deploy a security and compliance system"
      + resources   = (known after apply)
      + updated_at  = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Plan looked good, time for a run.

Another neat thing about using TFC is that my TF plan|apply actions were shown live in the web UI as well as my terminal. It also keeps a history of these actions. This is less important maybe for a 1-person project but I see real value for teams.

$ tf apply
Running apply in the remote backend. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.

Preparing the remote apply...

To view this run in a browser, visit:
https://app.terraform.io/app/ljones/do-k8s-2021/runs/run-QnJoQ2wVpHnqt1iv

Waiting for the plan to start...

Terraform v1.0.11
on linux_amd64
Configuring remote state backend...
Initializing Terraform configuration...

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # digitalocean_project.do-k8s-2021 will be created
  + resource "digitalocean_project" "do-k8s-2021" {
      + created_at  = (known after apply)
      + description = "DigitalOcean Kubernetes Challenge 2021"
      + environment = "Development"
      + id          = (known after apply)
      + is_default  = (known after apply)
      + name        = "do-k8s-2021"
      + owner_id    = (known after apply)
      + owner_uuid  = (known after apply)
      + purpose     = "Deploy a security and compliance system"
      + resources   = (known after apply)
      + updated_at  = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.


Do you want to perform these actions in workspace "do-k8s-2021"?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

digitalocean_project.do-k8s-2021: Creating...
digitalocean_project.do-k8s-2021: Creation complete after 2s [id=3401883b-cbc0-4889-ad45-a8d30051cd28]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

I’m not going to show the plan/apply output every time, by the way. I figure it might be helpful for those that haven’t used it before though.

I checked in the DO console and… drum roll… success! Terraform hadn’t been lying to me after all.

Kubernetes (k8s)

It was about this point that I found out Kubernetes on DO is called DigitalOcean Kubernetes or DOKS, and this entire exercise should have been called doks-challenge-2021 or something like that. Oh well.

At this point I had:

• DO project ready
• TFC operational
• A mostly empty main.tf

terraform {
  backend "remote" {
    organization = "ljones"

    workspaces {
      name = "do-k8s-2021"
    }
  }
  required_providers {
    digitalocean = {
      source  = "digitalocean/digitalocean"
      version = "~> 2.16.0"
    }
  }
}

provider "digitalocean" {}

resource "digitalocean_project" "do-k8s-2021" {
  name        = "do-k8s-2021"
  description = "DigitalOcean Kubernetes Challenge 2021"
  purpose     = "Deploy a security and compliance system"
}

The next step on my journey was actually deploying a Kubernetes cluster. At this point I just started skimming through the DO documentation on k8s, particularly cluster creation. I saw there was a HA-cluster option but nah, this isn’t that serious. I read up a little on DO vpcs, regions, block storage and load balancers. Surprisingly⁵ they don’t have an Australian region, but they do have Singapore and as I’m on the western coast of Australia, that’s pretty much as good as the eastern coast where all the cloud providers host their datacentres.

Then it was back to the TF DO documentation, eventually leading to these updates to main.tf.

resource "digitalocean_project" "do-k8s-2021" {
  name        = "do-k8s-2021"
  description = "DigitalOcean Kubernetes Challenge 2021"
  purpose     = "Deploy a security and compliance system"
  resources = [
    "do:kubernetes:${digitalocean_kubernetes_cluster.doks.id}" # this is a bit tedious
  ]
}

resource "digitalocean_vpc" "do-k8s-2021" {
  name   = "do-k8s-2021"
  region = "sgp1"
}

data "digitalocean_kubernetes_versions" "doks" {
  version_prefix = "1.21."
}

resource "digitalocean_kubernetes_cluster" "doks" {
  name         = "do-k8s-2021"
  region       = "sgp1"
  auto_upgrade = true
  version      = data.digitalocean_kubernetes_versions.doks.latest_version

  vpc_uuid = digitalocean_vpc.do-k8s-2021.id

  maintenance_policy {
    start_time = "16:00"
    day        = "friday"
  }

  node_pool {
    name       = "autoscale-worker-pool"
    size       = "s-1vcpu-2gb" # 1 cpu, 2gb ram (1gb useable)
    auto_scale = true
    min_nodes  = 1
    max_nodes  = 3 # this is the maximum allowed nodes on my new account without requesting an increase
  }
}

Applying, which creates the cluster, took about 6 minutes for me though that may vary for you depending on region and node size. It worked without a hitch though, noice.

The last thing I wanted to do was get the new k8s cluster into TF, so I can deploy manifests or helm charts. More reading of documentation, though this is actually something I’m already doing over in AWS EKS land so there wasn’t much to learn.

• Added the kubernetes tf provider
• Added the helm tf provider
• Initialised tf again
• Tested creating/destroying a namespace via tf just to make sure everything was 👌

  required_providers {
    ...
    helm = {
      source  = "hashicorp/helm"
      version = "~> 2.4.0"
    }
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = "~> 2.6.0"
    }
  }

...

provider "kubernetes" {
  host                   = digitalocean_kubernetes_cluster.doks.kube_config[0].host
  token                  = digitalocean_kubernetes_cluster.doks.kube_config[0].token
  cluster_ca_certificate = base64decode(digitalocean_kubernetes_cluster.doks.kube_config[0].cluster_ca_certificate)
}

provider "helm" {
  kubernetes {
    host                   = digitalocean_kubernetes_cluster.doks.kube_config[0].host
    token                  = digitalocean_kubernetes_cluster.doks.kube_config[0].token
    cluster_ca_certificate = base64decode(digitalocean_kubernetes_cluster.doks.kube_config[0].cluster_ca_certificate)
  }
}

Falco

Here we are, the final chapter. It was time to install Falco. At this point I had:

• DOKS operational and ready to be interacted with via tf

You may have noticed that I’m still only using a single file, main.tf.

I’m a fan of keeping things together until splitting them out actually makes my life easier, but this is maybe a style no-no for terraform. Sometimes empty files are used just to indicate the absence of that type of resource, for example, an empty outputs.tf indicating a module doesn’t have outputs.

For a small greenfield project where I’m the only one working on it, it really doesn’t matter though.

I went digging around and found the helm repository and accompanying documentation. Looked like it should run a basic install without any tweaking of the chart values, so I left everything to default.

resource "helm_release" "falco" {
  name       = "falco"
  repository = "https://falcosecurity.github.io/charts"
  chart      = "falco"
  version    = "1.16.2"

  namespace        = "falco"
  create_namespace = true
}

The tf apply was successful, and accessing the k8s dashboard via the DO console, I could see Falco had installed and was operational!

* Setting up /usr/src links from host
* Running falco-driver-loader for: falco version=0.30.0, driver version=3aa7a83bf7b9e6229a3824e3fd1f4452d1e95cb4
* Running falco-driver-loader with: driver=module, compile=yes, download=yes
* Unloading falco module, if present
* Trying to load a system falco module, if present
* Looking for a falco module locally (kernel 4.19.0-17-cloud-amd64)
* Trying to download a prebuilt falco module from https://download.falco.org/driver/3aa7a83bf7b9e6229a3824e3fd1f4452d1e95cb4/falco_debian_4.19.0-17-cloud-amd64_1.ko
* Download succeeded
* Success: falco module found and inserted
Mon Nov 22 11:33:41 2021: Falco version 0.30.0 (driver version 3aa7a83bf7b9e6229a3824e3fd1f4452d1e95cb4)
Mon Nov 22 11:33:41 2021: Falco initialized with configuration file /etc/falco/falco.yaml
Mon Nov 22 11:33:41 2021: Loading rules from file /etc/falco/falco_rules.yaml:
Mon Nov 22 11:33:41 2021: Loading rules from file /etc/falco/falco_rules.local.yaml:
Mon Nov 22 11:33:42 2021: Starting internal webserver, listening on port 8765
11:33:42.054459000: Notice Container with sensitive mount started (user=daemon user_loginuid=0 command=container:9f7394213f5d k8s.ns=kube-system k8s.pod=do-node-agent-tf8t9 container=9f7394213f5d image=docker.io/digitalocean/do-agent:3 mounts=/proc:/host/proc::false:private,/sys:/host/sys::false:private,/:/host/root::false:rslave,/var/lib/kubelet/pods/a8eac910-fb2e-40ea-bc13-2cfd181d8750/volumes/kubernetes.io~projected/kube-api-access-ctq56:/var/run/secrets/kubernetes.io/serviceaccount::false:private,/var/lib/kubelet/pods/a8eac910-fb2e-40ea-bc13-2cfd181d8750/etc-hosts:/etc/hosts::true:private,/var/lib/kubelet/pods/a8eac910-fb2e-40ea-bc13-2cfd181d8750/containers/do-node-agent/21e37c0b:/dev/termination-log::true:private) k8s.ns=kube-system k8s.pod=do-node-agent-tf8t9 container=9f7394213f5d
...

Thing is, nothing much was going on really. I had an essentially empty cluster at this point, so that made sense, but still! Happily, the falco helm chart actually has the ability to generate fake events. To do so, I added the below.

resource "helm_release" "falco" {
...
  set {
    name  = "fakeEventGenerator.enabled"
    value = "true"
  }
}

Tailing the logs once again, there was plenty going on now.

11:49:55.808634583: Warning Sensitive file opened for reading by trusted program after startup (user=root user_loginuid=-1 command=httpd --loglevel info run ^syscall.ReadSensitiveFileUntrusted$ --sleep 6s parent=httpd file=/etc/shadow parent=httpd gparent=event-generator container_id=64b9a21e112e image=docker.io/falcosecurity/event-generator) k8s.ns=falco k8s.pod=falco-event-generator-84d64cb8fb-j9tpl container=64b9a21e112e k8s.ns=falco k8s.pod=falco-event-generator-84d64cb8fb-j9tpl container=64b9a21e112e
11:49:55.911658311: Error File below a known binary directory opened for writing (user=root user_loginuid=-1 command=event-generator run --loop ^syscall file=/bin/created-by-event-generator parent=event-generator pcmdline=event-generator run --loop ^syscall gparent=<NA> container_id=64b9a21e112e image=docker.io/falcosecurity/event-generator) k8s.ns=falco k8s.pod=falco-event-generator-84d64cb8fb-j9tpl container=64b9a21e112e k8s.ns=falco k8s.pod=falco-event-generator-84d64cb8fb-j9tpl container=64b9a21e112e
11:49:55.911718964: Error File below known binary directory renamed/removed (user=root user_loginuid=-1 command=event-generator run --loop ^syscall pcmdline=event-generator run --loop ^syscall operation=unlinkat file=<NA> res=0 dirfd=-100(AT_FDCWD) name=/bin/created-by-event-generator flags=0  container_id=64b9a21e112e image=docker.io/falcosecurity/event-generator) k8s.ns=falco k8s.pod=falco-event-generator-84d64cb8fb-j9tpl container=64b9a21e112e k8s.ns=falco k8s.pod=falco-event-generator-84d64cb8fb-j9tpl container=64b9a21e112e
11:49:56.013072532: Notice System user ran an interactive command (user=bin user_loginuid=-1 command=login container_id=64b9a21e112e image=docker.io/falcosecurity/event-generator) k8s.ns=falco k8s.pod=falco-event-generator-84d64cb8fb-j9tpl container=64b9a21e112e k8s.ns=falco k8s.pod=falco-event-generator-84d64cb8fb-j9tpl container=64b9a21e112e
11:49:56.294084849: Debug Shell spawned by untrusted binary (user=root user_loginuid=-1 shell=bash parent=httpd cmdline=bash -c ls > /dev/null pcmdline=httpd --loglevel info run ^helper.RunShell$ gparent=httpd ggparent=event-generator aname[4]=event-generator aname[5]=<NA> aname[6]= aname[7]= container_id=64b9a21e112e image=docker.io/falcosecurity/event-generator) k8s.ns=falco k8s.pod=falco-event-generator-84d64cb8fb-j9tpl container=64b9a21e112e k8s.ns=falco k8s.pod=falco-event-generator-84d64cb8fb-j9tpl container=64b9a21e112e
11:49:56.501997391: Error File below /etc opened for writing (user=root user_loginuid=-1 command=event-generator run --loop ^syscall parent=event-generator pcmdline=event-generator run --loop ^syscall file=/etc/created-by-event-generator program=event-generator gparent=<NA> ggparent= gggparent= container_id=64b9a21e112e image=docker.io/falcosecurity/event-generator) k8s.ns=falco k8s.pod=falco-event-generator-84d64cb8fb-j9tpl container=64b9a21e112e k8s.ns=falco k8s.pod=falco-event-generator-84d64cb8fb-j9tpl container=64b9a21e112e
11:49:56.602813830: Error File created below /dev by untrusted program (user=root user_loginuid=-1 command=event-generator run --loop ^syscall file=/dev/created-by-event-generator container_id=64b9a21e112e image=docker.io/falcosecurity/event-generator) k8s.ns=falco k8s.pod=falco-event-generator-84d64cb8fb-j9tpl container=64b9a21e112e k8s.ns=falco k8s.pod=falco-event-generator-84d64cb8fb-j9tpl container=64b9a21e112e
11:49:56.703339933: Warning Sensitive file opened for reading by non-trusted program (user=root user_loginuid=-1 program=event-generator command=event-generator run --loop ^syscall file=/etc/shadow parent=event-generator gparent=<NA> ggparent= gggparent= container_id=64b9a21e112e image=docker.io/falcosecurity/event-generator) k8s.ns=falco k8s.pod=falco-event-generator-84d64cb8fb-j9tpl container=64b9a21e112e k8s.ns=falco k8s.pod=falco-event-generator-84d64cb8fb-j9tpl container=64b9a21e112e
11:49:56.963228274: Notice Database-related program spawned process other than itself (user=root user_loginuid=-1 program=ls parent=mysqld container_id=64b9a21e112e image=docker.io/falcosecurity/event-generator) k8s.ns=falco k8s.pod=falco-event-generator-84d64cb8fb-j9tpl container=64b9a21e112e k8s.ns=falco k8s.pod=falco-event-generator-84d64cb8fb-j9tpl container=64b9a21e112e
11:49:57.233889340: Notice Known system binary sent/received network traffic (user=root user_loginuid=-1 command=sha1sum --loglevel info run ^helper.NetworkActivity$ connection=10.244.0.83:49851->10.2.3.4:8192 container_id=64b9a21e112e image=docker.io/falcosecurity/event-generator) k8s.ns=falco k8s.pod=falco-event-generator-84d64cb8fb-j9tpl container=64b9a21e112e k8s.ns=falco k8s.pod=falco-event-generator-84d64cb8fb-j9tpl container=64b9a21e112e

Tailing the actual event generator pod, I saw corresponding logs.

INFO sleep for 100ms                               action=syscall.WriteBelowEtc
INFO writing to /etc/created-by-event-generator    action=syscall.WriteBelowEtc
INFO sleep for 100ms                               action=syscall.SystemUserInteractive
INFO run command as another user                   action=syscall.SystemUserInteractive cmdArgs="[]" cmdName=/bin/login user=daemon
INFO sleep for 100ms                               action=syscall.SystemProcsNetworkActivity
INFO spawn as "sha1sum"                            action=syscall.SystemProcsNetworkActivity args="^helper.NetworkActivity$"
INFO sleep for 100ms                               action=helper.NetworkActivity as=sha1sum
INFO action executed                               action=helper.NetworkActivity as=sha1sum
INFO sleep for 100ms                               action=syscall.ReadSensitiveFileTrustedAfterStartup
INFO spawn as "httpd"                              action=syscall.ReadSensitiveFileTrustedAfterStartup args="^syscall.ReadSensitiveFileUntrusted$ --sleep 6s"
INFO sleep for 6s                                  action=syscall.ReadSensitiveFileUntrusted as=httpd
INFO action executed                               action=syscall.ReadSensitiveFileUntrusted as=httpd
INFO sleep for 100ms                               action=syscall.WriteBelowBinaryDir
INFO writing to /bin/created-by-event-generator    action=syscall.WriteBelowBinaryDir

I didn’t actually need this running fake events forever, so I reverted the helm chart back to defaults and applied again.

Job done

That’s actually the challenge complete. As you can see, Terraform and managed Kubernetes providers like DigitalOcean make it pretty easy to get up and running quickly.

Considering I’ve still got a month or so to go, I’ll probably try some of the other challenges, or extend this one a bit further. Getting Loki and Grafana going with Falco sounds cool 🤔

Deploy a security and compliance system

When you deploy an image you may also deploy a security threat, and images can become quickly insecure as new threats are discovered and disclosed. To continuously monitor and detect threats, you can deploy a tool like Falco.

I’ll be making a separate post on my process and experience, which I’ll link to from here once complete. Update: here it is!

As a general comment, I’ve been really happy to see DO go from strength to strength. Competition in the marketplace is always¹ a good thing, and I think they were early arrivals in a now increasingly crowded group of low/lower-complexity alternatives to AWS.

If you’re interested in particpating too, use my referral code for an extra $100 credit. Not too shabby!

I’ll be posting each major exam area individually, to keep things human.

1. Compare authentication methods

1a. Describe authentication methods

• Auth methods are how Vault assigns identity/policy to a user
• Multiple auth methods available, covering many use cases
  • e.g. GitHub auth method is useful for people, AppRole is useful for machines
• Auth methods are enabled/disabled by CLI or API
• All auth methods are mounted under the /auth prefix, however the path can be customised
• The same auth method can be mounted multiple times
• After a client authenticates against an auth method, they are issued a token. The token may have attached policy. The policy is mapped at auth time
• It’s not possible to force a user to auth against multiple auth methods, though MFA is possible with some backends

# enable the userpass auth method
$ vault auth enable userpass
Success! Enabled userpass auth method at: userpass/

# enable the userpass auth method at another path
$ vault auth enable -path=my-login userpass
Success! Enabled userpass auth method at: my-login/

# an alternate way of enabling an auth method
$ vault write sys/auth/my-auth type=userpass
Success! Data written to: sys/auth/my-auth

# list auth methods
$ vault auth list
Path         Type        Accessor                  Description
----         ----        --------                  -----------
my-auth/     userpass    auth_userpass_23a812e6    n/a
my-login/    userpass    auth_userpass_57e8bdd5    n/a
token/       token       auth_token_8fe964a8       token based credentials
userpass/    userpass    auth_userpass_ae1dbd20    n/a

1b. Choose an authentication method based on use case

Q. A developer wants to login to the Vault UI to check that some secrets exist. All the developers in the organisation are using GitHub A. Use the GitHub auth method

Q. An operations engineer needs to configure a new auth method on a vault server. The organisation uses Microsoft AD to manage their staff identities A. Use the LDAP auth method

Q. A pod on Kubernetes needs to authenticate against Vault to pull secrets needed to run an application A. Use the Kubernetes auth method

1c. Differentiate human vs. system auth methods

• Human auth methods are typically intended to be accessed by the CLI or UI
• System auth methods typically use the API

Here ends part one! 1c. looks a little skinny to me but I really didn’t get much more from the docs. I’ll add to it if more comes up later.

Note I didn’t actually complete them before attempting the exam so you’ll still need to check the Hashicorp resources, but maybe they’ll still be of use to someone.

Good luck if that’s you and you’re planning to do the exam soon!

I use Terraform on a daily basis so while I’m on the certification bandwagon I may as well go with the obvious choices. This post will be my study guide, as mapped to the exam objectives.

Many thanks go to the comprehensive documentation and tutorials provided by Hashicorp, for Terraform and the rest of their products. ❤️

1. Understand Infrastructure as Code (IaC) concepts

1a. Explain what IaC is

Infrastructure as Code (IaC) is the concept of describing infrastructure (e.g. compute resources, networks, firewalls etc) in human-readable declarative configuration files.

1b. Describe advantages of IaC patterns

• Idempotent; safe to apply code multiple times without resulting in unwanted/duplicate infrastructure. If part of a deploy fails, you can safely resolve and apply again.
• Consistent; no opportunity for an engineer to accidentally miss a step while building out infrastructure.
• Repeatable; the same code can be applied across many environments, or used to build/tear down the same environment (e.g. dev environments).
• Predictable; only the indicated changes are applied.
• Commitable (is that even a word?); the IaC can be checked into version control systems (e.g. git)
  • Enables GitOps flows, where changes can automatically be applied provided that they pass tests.
  • Allows for pull requests, easy review of planned changes.
  • Easily view history of changes to infrastructure, simple to roll back if required.

2. Understand Terraform’s purpose (vs other IaC)

2a. Explain multi-cloud and provider-agnostic benefits

• Multi-cloud helps provide extreme HA/fault-tolerance across services/regions.
• Provider-agnostisic benefits
  • Single control plane for multi-cloud; action changes in multiple clouds with a single command.
  • Single consistent configuration syntax for operators to learn.

2b. Explain the benefits of state

• Maps resources to the real world, i.e. state allows Terraform to map resource "aws_instance" "foo" to i-abcd1234.
• Tracks metadata, e.g. dependency order between resources.
• Performance; while it’s possible to refresh the entire state per plan operation in smaller infrastructures, this becomes less feasible at scale due to the time required and potential API limits. To address this issue, the state file can be used as a cache via -refresh=false, and/or plan resources can be targeted/limited via -target.
• Syncing of state between users; use of a fully-featured state backend (e.g. S3 + DynamoDB, Consul) that ‘locks’ the state will prevent apply operations from occuring simultaneously. TL;DR, single point of truth.

3. Understand Terraform basics

3a. Handle Terraform and provider installation and versioning

Terraform installation

# Compiled from source
git clone https://github.com/hashicorp/terraform.git
cd terraform
go install

# Platform specific pre-compilied binaries available on the Terraform website
wget https://releases.hashicorp.com/terraform/1.0.3/terraform_1.0.3_darwin_amd64.zip

# Platform specific package management tools

## Homebrew on OS X
brew tap hashicorp/tap
brew install hashicorp/tap/terraform

## Chocolatey on Windows
choco install terraform

# Apt/yum/dnf on various Linux.

## Ubuntu/Debian
sudo apt-get update && sudo apt-get install -y gnupg software-properties-common curl
curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -
sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
sudo apt-get update && sudo apt-get install terraform

Terraform versioning

# versions.tf
terraform {
  required_version ">= 0.14"
}

• Provider definition and version definition

# main.tf
provider "aws" {
  region = "us-west-2"
}

# versions.tf
terraform {
  required_providers {
    aws = {
        source  = "hashicorp/aws"
        version = ">= 2.0.0"
    }
  }
}

Provider installation and versioning via .terraform.lock.hcl.

• Running init will download/install modules as necessary, and store specific versioning information in the .terraform.lock.hcl file.
  • This file should be version-controlled to ensure consistency across teams and ephemeral Terraform execution environments.
  • This file should not be directly modified.
• Running init -upgrade will upgrade all providers within given version constraints.

$ terraform init

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Reusing previous version of hashicorp/random from the dependency lock file
- Installing hashicorp/aws v2.50.0...
- Installed hashicorp/aws v2.50.0 (signed by HashiCorp)

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to seeany changes that are required for your infrastructure. All Terraform commands should now work.

$ cat .terraform.lock.hcl
# This file is maintained automatically by "terraform init".
# Manual edits may be lost in future updates.

provider "registry.terraform.io/hashicorp/aws" {
  version     = "2.50.0"
  constraints = ">= 2.0.0"
  hashes = [
    "h1:aKw4NLrMEAflsl1OXCCz6Ewo4ay9dpgSpkNHujRXXO8=",
    ## ...
    "zh:fdeaf059f86d0ab59cf68ece2e8cec522b506c47e2cfca7ba6125b1cd06b8680",
  ]
}

3b. Describe plug-in based architecture

• Terraform Core reads configuration files and builds up a resource dependency graph.
• Terraform plugins (i.e. providers and provisioners) bridge the Core to respective target APIs, .e.g. the AWS provider plugin implements S3 resources via calls to the S3 API.
• plan and update actions cause Core to ask the provider to perform an action via an RPC interface. The provider then attempts a CRUD action against the target API’s client library.
• Terraform is mainly used against cloud infrastructure but providers can be used to interface with any API to manage any resource.

3c. Demonstrate using multiple providers

# main.tf
provider "aws" {
  region = "us-west-2"
}

resource "random_pet" "petname" {
  length    = 5
  separator = "-"
}

resource "aws_s3_bucket" "sample" {
  bucket = random_pet.petname.id
  acl    = "public-read"
  region = "us-west-2"
}

# versions.tf
terraform {
  required_providers {
    random = {
      source  = "hashicorp/random"
      version = "3.0.0"
    }

    aws = {
        source  = "hashicorp/aws"
        version = ">= 2.0.0"
    }
  }
}

3d. Describe how Terraform finds and fetches providers

When init is run, Terraform will attempt to download providers from the Terraform registry.

• If a provider is not on the registry or the registry is known to be unreachable (e.g. due to corporate firewalls), you can override the default registry by providing an ‘Explicit Installation Method Configuration’ in the Terraform CLI config (.terraformrc or terraform.rc).
• If the default registry fails and an override hasn’t been provided, Terraform checks creates an implied override that checks for the provider in some default locations in the user path.

When the provider has been found, init will download the provider into a subdirectory of the working directory so that each working directory is self-contained. This can lead to unneccessary duplication, so a provider cache can be specified via plugin_cache_dir in the Terraform CLI config.

• Terraform will not create the cache directory for you.
• Terraform checks against the registry, then against the cache to see if it has a copy. If it does not, Terraform will download the plugin into the cache directory before copying it into the working directory. It will use symlinks where possible.
• The plugin cache directory cannot be referenced by an explicit or implied install configuration.
• The plugin cache directory does not self clean and will need to be manually cleaned over time.

3e. Explain when to use and not use provisioners and when to use local-exec or remote-exec

Provisioners can be used to model specific actions on the local machine or on a remote machine in order to prepare servers or other infrastructure objects for service.

Provisioners should be used a last resort.

• Terraform cannot model provisoners as part of a plan as they technically could be doing anything.
• Many more moving parts than are normally involved, e.g. direct network access to remote servers, credentials to login, potentially require pre-existing software to be installed on the remote.

Potential use cases for local-exec or remote-exec.

• When new functionality has been introduced to a service that is not yet implemented in their provider plugin.
• Passing data into virtual machines and other compute resources; however, often cloud services will provide their own mechanism for this e.g. user_data for AWS EC2.
• Running configuration management software; however, Hashicorp recommend a custom image build process instead e.g. through the use of Packer.

4. Use the Terraform CLI (outside of core workflow)

4a. Given a scenario: choose when to use terraform fmt to format code

The terraform fmt command is used to rewrite Terraform configuration files to a canonical format and style. This command applies a subset of the Terraform language style conventions, along with other minor adjustments for readability.

The format command scans the current directory for configuration files and rewrites your Terraform configuration files to the recommended format.

• fmt and the format/style applied should always be used, basically. It is intentionally opinionated to ensure consistency across teams and projects.
• The canonical format may have minor changes between versions so Hashicorp recommend running fmt after upgrading.
• fmt will detect errors in your code and is useful for validation

$ terraform fmt
Error: Invalid character

  on main.tf line 46, in resource "aws_instance" "web_app":
  46:     Name = $var.name-learn

This character is not used within the language.

4b. Given a scenario: choose when to use terraform taint to taint Terraform resources

The terraform taint command informs Terraform that a particular object has become degraded or damaged. Terraform represents this by marking the object as “tainted” in the Terraform state, in which case Terraform will propose to replace it in the next plan you create.

Replacing a resource is also useful in cases where a user manually changes a setting on a resource or when you need to update a provisioning script. This allows you to rebuild specific resources and avoid a full terraform destroy operation on your configuration.

taint is deprecated, instead use -replace="resource" with plan or apply commands. The latter will allow you to see the full effect of the change, i.e. tainting a single resource may force the recreation of one or more other resources, which will be visible in the plan.

4c. Given a scenario: choose when to use terraform import to import existing infrastructure into your Terraform state

The terraform import command is used to import existing resources into Terraform.

import helps Terraform assume control over existing infrastructure or resources otherwise created outside of Terraform (and thus are not in the state file). This is as opposed to manually editing the state file which is not recommended.

Through import, Terraform can generate the appropriate code to represent your existing resources.

# Create an empty resource in terraform
$ cat main.tf
resource "docker_container" "web" {}

# Import the existing resource into terraform
terraform import docker_container.web $(docker inspect --format="{{.ID}}" hashicorp-learn)

# Check the state contains the resource as expected
$ terraform show
# docker_container.web:
resource "docker_container" "web" {
  command = [
    "nginx",
    "-g",
    "daemon off;",
  ]
# ...

# Generate configuration from the state
terraform show -no-color > main.tf
$ cat main.tf
# docker_container.web:
resource "docker_container" "web" {
  command = [
    "nginx",
    "-g",
    "daemon off;",
  ]
# ...

Note that dumping the state this way may contain attributes that cannot actually be assigned via the configuration. Be sure to run a plan to detect any issues and modify the resource as required.

4d. Given a scenario: choose when to use terraform workspace to create workspaces

Each Terraform configuration has an associated backend that defines how operations are executed and where persistent data such as the Terraform state are stored.

The persistent data stored in the backend belongs to a workspace. Initially the backend has only one workspace, called “default”, and thus there is only one Terraform state associated with that configuration.

Certain backends support multiple named workspaces, allowing multiple states to be associated with a single configuration. The configuration still has only one backend, but multiple distinct instances of that configuration to be deployed without configuring a new backend or changing authentication credentials.

• Workspaces can be used to create multiple instances of infrastructure from the same Terraform configuration.
• Workspaces are less suitable for separating environments, e.g. development from production, as each environment should have its own configuration, backend and workspaces.
  • They may be appropriate in this case, in the interests of avoiding duplicating code, if there is very little drift between the environments.
  • When different credentials and backends are required between environments, splitting environments by using directories is preferable, at the risk of duplicating code and allowing drift to creep in. Modules should be used to address both those issues, however.
• Workspaces are more suitable for creating short-lived infrastructures used during development, e.g. a developer might branch the main trunk, create a new workspace for testing/developing that branch and then delete the workspace after the branch is merged into main.
• Workspaces can be referenced as variables, .e.g. count = "${terraform.workspace == "default" ? 5 : 1}"

4e. Given a scenario: choose when to use terraform state to view Terraform state

Terraform stores information about your infrastructure in a state file. This state file keeps track of resources created by your configuration and maps them to real-world resources.

The terraform state command is used for advanced state management. As your Terraform usage becomes more advanced, there are some cases where you may need to modify the Terraform state. Rather than modify the state directly, the terraform state commands can be used in many cases instead.

• state commands work the same with local and remote state backends.
• All modifying state commands create a backup file, and this behaviour cannot be disabled.
• state has been designed to be friendly with other CLI tools.

Really, state should be used for all state file interactions.

4f. Given a scenario: choose when to enable verbose logging and what the outcome/value is

Terraform has detailed logs which can be enabled by setting the TF_LOG environment variable to any value. This will cause detailed logs to appear on stderr.

• You can set TF_LOG to one of the log levels TRACE, DEBUG, INFO, WARN or ERROR to change the verbosity of the logs.
• Setting TF_LOG to JSON outputs logs at the TRACE level or higher, and uses a parseable JSON encoding as the formatting.
• Logging can be enabled separately for terraform itself and the provider plugins using the TF_LOG_CORE or TF_LOG_PROVIDER environment variables. These take the same level arguments as TF_LOG, but only activate a subset of the logs.
• To persist logged output you can set TF_LOG_PATH in order to force the log to always be appended to a specific file when logging is enabled. Note that even when TF_LOG_PATH is set, TF_LOG must be set in order for any logging to be enabled.

Verbose logging is useful when you’re encountering issues with either Terraform Core or a plugin. In particular, it’s expected you’ll use these logs to determine where a crash or issue is caused, before opening an issue on the appropriate github repository. The TRACE logging level should be used for these logs.

5. Interact with Terraform modules

5a. Contrast module source options

The Terraform Registry is integrated directly into Terraform, so a Terraform configuration can refer to any module published in the registry.

Each module in the registry is versioned. These versions syntactically must follow semantic versioning. In addition to pure syntax, we encourage all modules to follow the full guidelines of semantic versioning.

• Module sources.
  • Terraform Registry modules.
    • The default registry.
    • Only shows verified modules by default.
    • Source strings of the form <NAMESPACE>/<NAME>/<PROVIDER>. For example: hashicorp/consul/aws.
  • Private Registry modules.
    • Terraform Cloud provides a private registry, as an example.
    • Ideal for sharing modules within or across an organisation.
    • Public modules from the Terraform Registry can be added to a Private Registry to provide a curated list of approved modules.
    • May need to provide credentials for access.
    • Source strings of the form <HOSTNAME>/<NAMESPACE>/<NAME>/<PROVIDER>. This is the same format as the public registry, but with an added hostname prefix.
  • Local modules.
    • Terraform treats any local directory referenced in the source argument of a module block as a module.

5b. Interact with module inputs and outputs

• Input variables are like function arguments.
• Output values are like function return values.
• Local values are like a function’s temporary local variables.

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "2.21.0"

  # everything below this point is a module input
  name = var.vpc_name
  cidr = var.vpc_cidr

  azs             = var.vpc_azs
  private_subnets = var.vpc_private_subnets
  public_subnets  = var.vpc_public_subnets

  enable_nat_gateway = var.vpc_enable_nat_gateway

  tags = var.vpc_tags
}

For the given module inputs above, there are corresponding input variables inside the module.

variable "name" {
  description = "Name to be used on all the resources as identifier"
  type        = string
  default     = ""
}

variable "cidr" {
  description = "The CIDR block for the VPC. Default value is a valid CIDR, but not acceptable by AWS and should be overridden"
  type        = string
  default     = "0.0.0.0/0"
}

The resources defined in a module are encapsulated, so the calling module cannot access their attributes directly. However, the child module can declare output values to selectively export certain values to be accessed by the calling module.

output "vpc_id" {
  description = "The ID of the VPC"
  value       = concat(aws_vpc.this.*.id, [""])[0]
}

This output could then be access via module.vpc.vpc_id.

5c. Describe variable scope within modules/child modules

Within the module that declared a variable, its value can be accessed from within expressions as var.<NAME>, where <NAME> matches the label given in the declaration block

The value assigned to a variable can only be accessed in expressions within the module where it was declared.

TL;DR, variables are scoped to the module they are declared in and need to redeclared in child modules if they need to be passed down.

Locals can be useful for merging values or other variables. Alternatively they can be used to create short references to otherwise long variables or values.

locals {
  # Ids for multiple sets of EC2 instances, merged together
  instance_ids = concat(aws_instance.blue.*.id, aws_instance.green.*.id)

  # merging variables
  name-prefix = "${var.project_name}-${var.environment}"

  # shortening a long value
  vpc_id = data.terraform_remote_state.test_project.outputs.vpc_id
}

resource "aws_s3_bucket" "default" {
  bucket = "${local.name-prefix}-bucket"
  acl    = "private"

  tags = {
    Name = "${local.name-prefix}-bucket"
  }
}

5d. Discover modules from the public Terraform Module Registry

I don’t really know what to put for this one. Browse and get comfortable with https://registry.terraform.io/browse/modules.

5e. Defining module version

• Hashicorp recommend explicitly constraining the acceptable version numbers to avoid unexpected or unwanted changes.
  • The following operators are valid:
  • = (or no operator): Allows only one exact version number. Cannot be combined with other conditions.
  • !=: Excludes an exact version number.
  • >, >=, <, <=: Comparisons against a specified version, allowing versions for which the comparison is true. “Greater-than” requests newer versions, and “less-than” requests older versions.
  • ~>: Allows only the rightmost version component to increment. For example, to allow new patch releases within a specific minor release, use the full version number: ~> 1.0.4 will allow installation of 1.0.5 and 1.0.10 but not 1.1.0. This is usually called the pessimistic constraint operator.
• Terraform will use the newest installed version of the module that meets the constraint; if no acceptable versions are installed, it will download the newest version that meets the constraint.
• Version constraints are supported only for modules installed from a module registry, such as the public Terraform Registry or Terraform Cloud’s private module registry. Other module sources can provide their own versioning mechanisms within the source string itself, or might not support versions at all. In particular, modules sourced from local file paths do not support version; since they’re loaded from the same source repository, they always share the same version as their caller.

• Less than and equal to operators can be changed to create an acceptable range, e.g. >= 1.2.0, < 2.0.0

module "consul" {
  source  = "hashicorp/consul/aws"
  version = "0.0.5"

  servers = 3
}

6. Navigate Terraform workflow

6a. Describe Terraform workflow ( Write -> Plan -> Create )

The core Terraform workflow has three steps:

• Write - Author infrastructure as code.
• Plan - Preview changes before applying.
• (version control commit)
• Apply - Provision reproducible infrastructure.
• (version control push)

When working with Terraform as an individual, these can all happen locally and have changes pushed to version control at the end of the cycle. As teams grow and managing secrets/credentials becomes more difficult, it’s normal to introduce a CI system. You can then lower individuals access to sensitive details while still allowing them to work with them. It also allows for branching/merge-request workflows where team members can review and approve changes before they are applied. An example flow might look like the below.

• (version control branch)
• Write code.
• (version control commit)
• Write code.
• (version control commit)
• (version control push)
• CI Plan
• (team-member reviews plan output and approves)
• (version control merge)
• CI Plan
• (team-member reviews merged plan output and approves)
• CI Apply

This can add time to the development workflow so be pragmatic and build what suits your company and situation.

6b. Initialize a Terraform working directory (terraform init)

When you create a new configuration — or check out an existing configuration from version control — you need to initialize the directory with terraform init. Initializing a configuration directory downloads and installs the providers defined in the configuration, which in this case is the aws provider.

$ terraform init

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Reusing previous version of hashicorp/random from the dependency lock file
- Installing hashicorp/aws v2.50.0...
- Installed hashicorp/aws v2.50.0 (signed by HashiCorp)

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to seeany changes that are required for your infrastructure. All Terraform commands should now work.

6c. Validate a Terraform configuration (terraform validate)

The terraform validate command validates the configuration files in a directory, referring only to the configuration and not accessing any remote services such as remote state, provider APIs, etc.

Validate runs checks that verify whether a configuration is syntactically valid and internally consistent, regardless of any provided variables or existing state. It is thus primarily useful for general verification of reusable modules, including correctness of attribute names and value types.

It is safe to run this command automatically, for example as a post-save check in a text editor or as a test step for a re-usable module in a CI system.

Validation requires an initialized working directory with any referenced plugins and modules installed.

• plan includes an implied validation check.

6d. Generate and review an execution plan for Terraform (terraform plan)

The terraform plan command creates an execution plan. By default, creating a plan consists of:

• Reading the current state of any already-existing remote objects to make sure that the Terraform state is up-to-date.
• Comparing the current configuration to the prior state and noting any differences.
• Proposing a set of change actions that should, if applied, make the remote objects match the configuration.

You can use the optional -out=FILE option to save the generated plan to a file on disk, which you can later execute by passing the file to terraform apply as an extra argument. This two-step workflow is primarily intended for when running Terraform in automation. If you run terraform plan without the -out=FILE option then it will create a speculative plan, which is a description of a the effect of the plan but without any intent to actually apply it. In teams that use a version control and code review workflow for making changes to real infrastructure, developers can use speculative plans to verify the effect of their changes before submitting them for code review. However, it’s important to consider that other changes made to the target system in the meantime might cause the final effect of a configuration change to be different than what an earlier speculative plan indicated, so you should always re-check the final non-speculative plan before applying to make sure that it still matches your intent.

6e. Execute changes to infrastructure with Terraform (terraform apply)

If you are using Terraform directly in an interactive terminal and you expect to apply the changes Terraform proposes, you can alternatively run terraform apply directly. By default, the “apply” command automatically generates a new plan and prompts for you to approve it.

6f. Destroy Terraform managed infrastructure (terraform destroy)

Okay so I mentioned in a recent post that I was going to write up how I used GoReleaser, Drone CI and Gitea to release a Golang project. I’ve decided to go one better and create a demo project for you to try it out yourself. Before getting into it though, there’s something we need to discuss.

The Drone CI documentation on Gitea mentions the following.

Please note we strongly recommend installing Drone on a dedicated instance. We do not recommend installing Drone and Gitea on the same machine due to network complications, and we definitely do not recommend installing Drone and Gitea on the same machine using docker-compose.

Network complications? 🤔 Well, it’s to do with the different networks at play. When you fire up docker-compose, it automatically creates a little dedicated network where you can resolve the other services by their name, no sweat.

Here’s a snippet of the compose.yaml.

services:
  gitea:
    ...

  drone:
    environment:
      - DRONE_GITEA_SERVER=http://gitea:3000
    ...

  drone-runner:
    image: drone/drone-runner-docker:1
    container_name: drone-runner
    environment:
      - DRONE_RPC_HOST=drone

And here’s the network created.

$ docker compose up -d
$ docker network ls
NETWORK ID     NAME                                  DRIVER    SCOPE
d945699fb57c   go-goreleaser-drone-example_default   bridge    local

You can see that service referencing at play twice in the compose.yaml, in the environment variables. http://gitea:3000 will resolve to the IP address of the Gitea container created. So where’s the problem? Well, the Drone CI Docker Runner creates containers to run your CI/CD pipeline. These containers are not inside the docker-compose network, so they can’t resolve http://gitea:3000. See below for the sad-face experience when a runner begins with the above config.

Initialized empty Git repository in /drone/src/.git/
+ git fetch origin +refs/heads/master:
fatal: unable to access 'http://gitea:3000/gitea/test.git/': Failed to connect to gitea port 3000: Connection refused

How can we work around this?

One, you could provide the IP address of the host running docker-compose in the compose.yaml. Note you only need to swap out the references to the Gitea service.

services:
  gitea:
    ...

  drone:
    environment:
      - DRONE_GITEA_SERVER=http://${HOST_IP}:3000
    ...

  drone-runner:
    image: drone/drone-runner-docker:1
    container_name: drone-runner
    environment:
      - DRONE_RPC_HOST=drone

Two, you could edit the /etc/hosts¹ config on the host running docker-compose to resolve the service names to the docker host, or the IP address of the docker desktop host.

$ ip a
...
5: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:15:5d:b0:24:13 brd ff:ff:ff:ff:ff:ff
    inet 172.28.200.90/20 brd 172.28.207.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::215:5dff:feb0:2413/64 scope link
       valid_lft forever preferred_lft forever

172.28.200.90 is the IP address I need to add to the hosts file. I’ve also added drone here so that I can reach it easily from a browser at http://drone:8080.

172.28.200.90 gitea drone

I prefer the second method but it’s really up to you. The first method is better if you’re going to be accessing from multiple hosts, as you don’t want to have to update the hosts file on every machine. This is just a demo, so the latter is fine.

Okay, detour complete, in the next post we’ll actually start bringing things online!

So aside from work, what’s been going on? Well, I’ve been fairly busy with some personal projects, one in particular helping out the father-in-law¹ with an integration to FileMaker(FM). He’s a smart guy and FileMaker is a low-code solution that lets him prototype and get the 80% achieved very quickly, but there are some rough edges. In this case, he’s calling out to an API to send SMS and wants to get replies back into the database, but FM doesn’t readily provide a callback target.

In a couple of hours² I had put together a little Golang program that effectively acts as a proxy, receiving the callbacks from the SMS API and writing them into the FM database. This is something I’ll probably open-source once I’m ready. I then spent the next two to three days going way overboard, refreshing myself on GoReleaser, learning Drone CI/CD and stitching it all together on my self-hosted Gitea instance. So now I can tag a release and it automatically does-the-needful™️. I’m planning on dedicating a post to how that all works, with an accompanying example repo.

Time for brunch now though, till next time!

In other news, BSIDES Perth was on the weekend just past. Some cool talks but personally it was just nice to see a bunch of familiar faces; former coworkers from previous security roles etc. Hat tip to the organisers, trying to get anything of this nature done in COVID times has to be just that much harder.

Heard 👂

Podcasts 🎤

• The Changelog: The business model of open source
  • Worth a listen if you’re planning on running a business based on open source software.
• The Changelog: Coding in the cloud with Codespaces
  • How Codespaces was sold to internal users within GitHub was pretty cool, trying to make it more of an exciting exclusive club rather than dogfooding.
• Ship It!: Real-world implications of shipping many times a day
• Go Time: Books that teach Go
  • Definitely a few things here for me to check out.

Music 🎵

• Slay King by Alter Boy
• second margin knife by nervous_testpilot
• cheap groovebox by _ensnare_
• Infinite Mirror - feat. Tomotaka Tsuji by LITE, Tomotaka Tsuji
• Kiwi by Seb Zillner, Lazlow
• Save Room by Waveshaper
• Last Jungle (Synth System Remix) by Sub Focus, Synth System
• LAE by Uppermost, FA
• Pound a Rhythm Electronic by Younger Brother
• Contact Note by Jon Hopkins
• Touching The Touchstone by Bill Laurance
• afterwards by yutaka hirasaka
• Digital Baptism by Falconite
• Aequilibrium by Andrey Vinogradov
• Greenpath by Christopher Larkin
• Lonely Rolling Star (Missing You) by aivi & surasshu
• Leaning In by Blue States
• Wrinkle by Thank You Scientist
• Watch the Sun by Groundislava

Watched 👀

YouTube 📺

• IFSC World Championships Moscow 2021 || Lead Semi-finals
  • Such skilled climbers!
• IFSC World Championships Moscow 2021 || Men’s Boulder Final
  • Such skilled boulderers! That’s a word! Probably 😬

Played 🎮

PC 💻

• RimWorld

Read 📜

Articles 📰

• The Uber Go Style Guide
• NSA & CISA Kubernetes Security Guidance – A Critical Review
• Clash: A rule-based tunnel in Go

Heard 👂

Podcasts 🎤

• Founders Talk: Iterating to globally distributed apps and databases
  • I really like the points about What are you optimising for? and The Velvet Rope.
  • The former is something that you can be asking yourself as a business but also just as a person living their life. What do you want to be achieving in life? Is that congruent with what you’re currently doing? If not, why not? Basically a reminder to step back and think about the bigger picture.
  • The latter is sort of in-line with the previous point, but basically using the analogy of being a bouncer at a bar. What kind of clientelle do you want to serve? Do you want to service a single high-paying customer or do you want to offer something less bespoke that’s affordable?
  • As a kind of aside, I’m a big fan of businesses doing one thing well, rather than trying to be everything to anyone. That said, I understand and have seen how it can be difficult when you have existing customers asking for features, possibly ones that your competitors are already offering or introducing, and there’s a pressure to keep up. Do you follow the competition and dilute your offering?
• Ship It!: Bare metal meets Kubernetes
• Go Time: Building actually maintainable software

Music 🎵

Really didn’t listen to much music this week.

• Friends in Dreams by The American Dollar, Linearwave
• Stomp (Bastion Remix) by Sub Focus, Bastion
• Example by No Mana, Uppermost
• Chronos - Live by Nu Deco Ensemble, Aaron Parks

Watched 👀

Movies 🎬

• The Suicide Squad (2021)
  • It’s a passable action movie. Set your expectations for mild entertainment and you might not be disappointed.

Read 📜

Articles 📰

• Why every software engineering interview should include ops questions
  • Charity Majors is a treasure.
• Cloud Native Security Map
• praetorian-inc/gokart
• A Guide to Interfaces in Go

Heard 👂

Podcasts 🎤

• Go Time: To build, or to buy, that is the question
• Ship It: Docs are not optional

Music 🎵

• Curicó by Kiltro
• All That’s Left (feat. Joni Fatora) - The M Machine Remix by Manila Killa, Joni Fatora, The M Machine
• The Great Calamity by Challenger Deep
• Thirty Seven After Six by Valve
• vacillate by Oliver Buckland
• Colored Engine by Joel Schoch
• Refuge by Dan Terminus
• Sun by Love Supreme
• This Means War - Feather & Leviathan Owl Remix by Volkor X, Feather, Leviathan Owl
• Adrenaline by Shirobon
• click by C418
• Beginnings by Uppermost
• Nate Smith Is the Ace of Aces by The Fearless Flyers
• Birds Canticum “Birds Requiem” Suite by Dhafer Youssef

Watched 👀

TV 📺

• The Simpsons
  • Over the year, I’ve been rewatching the golden oldies from season 2 onwards. I’m currently up to season 7. There’s a weaker episode here and there but I still find it to be very solid.

Movies 🎬

• The Dawn Wall
  • Utter madness. Human perseverance in a true story that goes way beyond rock climbing.

Played 🎮

PC 💻

• QUAKE
  • By some wierd coincidence, I reinstall this game just as they release a remaster. I’d never really played through the whole thing before, but this time I found myself getting into it in a way that I hadn’t before. The remaster is really well done and strikes that nostalgia balance of Ahh, just as I remember! :D and not Oof, just as I remember! D:. If you’ve never given it a go, there’s never been a better time.

Read 📜

Books 📖

• [Ender’s Saga #3: Xenocide][xeno] (finished)
  • I was losing interest but pushed through to the end. There’s a lot of squabbling/in-fighting between characters and I think that just got tedious for me.

Articles 📰

• PostgreSQL vs Redis vs Memcached performance
  • PostgreSQL performing pretty well. An interesting note in the comments about the network as well.
• Upgrade your Amazon RDS for PostgreSQL database, Part 1: Comparing upgrade approaches and Part 2
  • I may need to do the replication method at some stage in the future so it’s handy having this reference.

Heard 👂

Podcasts 🎤

• Go Time: Don’t forget about memory management
  • I always find these discussions of how things are operating at a lower level to be interesting, even if they’re not always applicable to anything I’m doing.
• The Changelog: Building software for yourself
  • Inspiring that this person has built their own language and surrounding ecosystem.

Music 🎵

• Dire, Dire Docks (Super Mario 64) by GENTLE LOVE
• Coming Home to Luncheon by Lena Raine
• Farewell by Lena Raine
• Joy of Remembrance by Lena Raine
• Mistake by Orax
• Losing Grip by CRT_HEAD
• Breakdown by Droid Bishop
• HALOHEAD by KUNZITE, Ratatat
• Aquatic Ambiance (Donkey Kong Country) by GENTLE LOVE
• Wallwalker by Maserati
• King’s Filth by Le Matos
• Machine Guns and Peacock Feathers - Carpenter Brut Remix by Ulver, Carpenter Brut
• Prism by Dugo
• Attack Ships On Fire by London Elektricity
• Deckard Returns by Chrome Canyon
• Pursuer by Sound of Ceres

Watched 👀

Movies 🎬

• Lady Bird
  • Half coming-of-age story, half love-letter to Sacramento. I enjoyed it.

Read 📜

Books 📖

• Ender’s Saga #3: Xenocide (in-progress)
  • About a third of the way, I’m enjoying it so far.

Articles 📰

• An Old Hacker’s Tips On Staying Employed

So the Two-And-Done rule was born, wherein I will state my case the first time, and if whoever is arguing to the contrary does not agree after hearing my position, I’ll let it go. But the next time the opportunity comes up, I will argue my point again. Maybe allowing for a gap of time for people to consider my original point, or maybe allowing me time to refine and rephrase my ideas to be more convincing.

If I fail to get my way after the second time though, I am done. I will even say as much to whomever I am debating if they are the final decision-maker. I will say something like, “OK, let’s go your way then. I still don’t completely agree with everything proposed here, but I think I’ve made my case, and we need to move on.”

• Common Anti-Patterns in Go Web Applications
• DevOps Engineer Crash Course - Section 1
  • Yep, quite familiar.

You cannot become an expert on every element you interact with. A day in my job can be managing Postgres operations, writing a PR against an application, or sitting in a planning session helping to design a new application. The scope of what many businesses call “DevOps” is too vast to be a deep-dive expert in all parts of it.

Heard 👂

Podcasts 🎤

• Modern CTO: #365 Shiven Ramji – Keeping Engineers Close to Customers, & The Passwordless Future
• JS Party: We ask a lawyer about GitHub Copilot
• Go Time: Caddy V2
  • Interesting to hear about Caddy in general, as it was a tool I got into early on and put to great use in production.
• Ship It! Assemble all your infrastructure
  • I’m going to have to look into it more, could be useful but there’s just so much going on in this space.
• Go Time: Data streaming and Benthos

Music 🎵

• Better Than Reality by Luklash
  • Rocking synths and chiptune breakdowns. The first track, Perpetual Motion sets the stage.
• Celeste B-Sides by Lena Raine
  • Loved the game and the soundtrack. These alternate mixes of tracks work well for me. Great for study/coding.
• Mountain Side by Beardyman
  • Very chill.

Watched 👀

TV 📺

• Queen’s Gambit S1E4-7
  • Yeah I quite enjoyed this one overall. Well produced and acted. I should try the book and see how I think it compares.

Movies 🎬

• A Quiet Place Part II
  • I remember enjoying part one though not specifically why. Maybe characters weren’t making obviously awful decisions? I didn’t enjoy this movie.

Articles

• Simple Systems Have Less Downtime.
• Discovery When Working Remotely.
• S3 backups and other strategies for ensuring data durability through ransomware attacks.
• AWS Organizations - Checklist for 2021.

Heard

Podcasts

• Developer Tea: Joel Beasley from Modern CTO
  • Host Jonathan Cutrell, director of technology at PBS, interviews and is interviewed by fellow podcaster Joel Beasley on life and work in general.
  • First advice when anyone asks for guidance: focus on your health. Work on your sleep, eating and exercise above everything else if you want to be a better developer.
• Go Time: Opening up the opinion box
  • Least/most popular ‘unpopular’ opinions collected from the regular show segment.
  • Probably can skip listening to this one unless you have a particular interest in the show or Go. It’s a bit of a filler episode.
• ABC Conversations: The Chloroformist — extraordinary Doctor Joseph Clover
• Open Source Security: Episode 283 – When vulnerability disclosure becomes dangerous
  • What do you do when you find an open database full of sensitive, potentially dangerous Personally Identifiable Information (PII)? If you report it and the company responsible doesn’t do anything, what next? There’s no easy answer as going public may cause more harm than good.
• The Changelog: Leading leaders who lead engineers

Music

• Playback by </body>
  • Hitting me somewhere in those vapor/synth-wave feels.

Watched

TV

• Queen’s Gambit S1E2-3
  • It’s quite entertaining. I had thought it was based in some way on a real person but apparently not.

Movies

• The Omega Man
  • Can’t say I really recommend it.
• Yojimbo
  • Classic samurai film.

Online

• Expectations vs Agreements
  • About 30 minutes on the value of coming to agreements with people rather than dictating expectations.
  • I need this by Friday, get it done. vs I need this by Friday, do you have the resources to make that happen?.
  • Opening a dialogue, or more broadly, communication is key.
  • Empowering people and giving them a sense of control is key.
  • This applies to every relationship in your life, not just those in the workplace.
• The Tale of Tiffany
  • Origins of the name Tiffany.
• System Shock 2 Review
  • I enjoy MandaloreGaming’s deep dive reviews. I particularly enjoyed discovering the overlaps between SS2 and the Thief series.

• Here’s what doesn’t work, in the workplace. Expectations. People do not look forward to living up to expectations. In fact, people rebel against expectations. They don’t like it that you expect things of them. They would rather you agree with them about something.
• Human beings will love to keep an agreement that they coauthored with you.

Read

Books

Ender’s Game / Speaker for the Dead

Somehow I hadn’t touched this series before, while catching parts of the movie. I quite enjoyed them both, with a slight preference for Ender's Game. I think the tighter focus and scope made it overall a more coherent narrative. Hard to follow up, certainly.

Normal People

I churned through this one pretty quickly, it’s easy reading though somewhat painful. Consider it a masterclass in how to fail at communication and hurt the people you care most about. I guess I found it engaging because like a train wreck in slow-motion, I couldn’t quite look away.

Atomic Habits

Like most self-improvement books, it’s generally things you already inherently know but either haven’t thought about or haven’t built a framework around to actually implement. This book provides the latter with some explanation around why it should work. As usual, ain’t nothing to it but to do it.

Heard

Podcasts

Ship It #11: Honeycomb’s secret to high-performing teams

Charity Majors seems to strike the right balance of take-no-prisoners confidence while welcoming alternative opinions. Bring them if you’ve got them, but I know what I’m talking about so you better have a good argument, that kind of jazz. Anyway, if you have any interest in monitoring or observability I think she’s someone you should listen to.

ABC Conversations

• Taming the Black Dog, and burnout
• Rebel doctor Caroline de Costa — smuggling condoms and scaring priests

Both interesting and worth listening to, for quite different reasons. The former I found interesting as someone who has experienced burnout in the past, the latter because Caroline de Costa just got up to so much! I learned a few things too.

Did you know that rabbits were used in human pregnancy tests? I didn’t!

Music

Bill Laurance

The first track I heard was The Good Things and I think I proceeded to listen to about three albums immediately thereafter. The transition at around 4:25 on that track is a killer. Jazzy, interesting, funky, exciting, background/focus/study music? Some of those words don’t work together but this music sure does for me.

Watched

TV

Olympic Sport Climbing

Bouldering about three times a week with a bit of lead climbing makes up the bulk of the exercise I get, and I love it. I’ve never really been a sporting person though I’ve gone through phases of weight-lifting, but rock climbing works really well for me. The physical and the mental/problem-solving challenge is just 👌.

Anyway, watching some of the best climbers in the world do their thing? Heck yes.

Movies

Booksmart

It’s a smarter, better Superbad, basically. Shout out to this Superbad scene though, while the rest of it has probably aged poorly this still gets chuckles from me. 🤷‍♂️

That’s a wrap, it was worthwhile. 😅

There’s three ways you could achieve this:

• You can look it up dynamically, as I have. This assumes you have access to credentials which are able to query that AWS account.
• If the other VPC is also managed by Terraform, you could query the state file for this information. You’d need access to the state file, obviously.
• You could look up (or be given) the IPs and just hardcode them or add them as a variable. NAT gateways are unlikely to change frequently unless the infrastructure is frequently being torn up/down so while manual, it would work.

Each of the steps below corresponds to a block of Terraform code.

1. Lookup the other VPC, in this came filtering on name vpc.
2. Lookup all the elastic network interfaces in the other account that are;
   1. In the above VPC,
   2. Have source destination checking disabled (required by NAT Gateways and NAT instances),
   3. Requester-managed, which means it’s a network interface that was created by an AWS service.
3. Collect data on each of the IPs that we got in the previous step. You’re not ready to use these wherever you need.
4. As an example, you could use them in an AWS WAF IP set.

# 1
data "aws_vpc" "other_account" {
  provider = aws.other_account

  tags = {
    Name = "vpc"
  }
}

# 2
data "aws_network_interfaces" "other_account_vpc_ngw_enis" {
  provider = aws.other_account

  filter {
    name   = "vpc-id"
    values = [data.aws_vpc.other_account.id]
  }
  filter {
    name   = "source-dest-check"
    values = ["false"]
  }
  filter {
    name   = "requester-managed"
    values = ["true"]
  }
}

# 3
data "aws_network_interface" "eni_ids" {
  provider = aws.other_account

  for_each = data.aws_network_interfaces.other_account_vpc_ngw_enis.ids
  id       = each.value
}

# 4
resource "aws_wafv2_ip_set" "allowlist" {
  name               = "allowlist"
  scope              = "REGIONAL"
  ip_address_version = "IPV4"
  addresses = [for eni in data.aws_network_interface.eni_ids : "${eni.association[0].public_ip}/32"]
}

It feels a little bit clunky to me at the moment, but it works. 👌

This certification validates the ability to design and implement distributed systems on AWS.

Basically, it says you have a clue of what to do with AWS. That’s not to say it’s easy; AWS has many, many services and you’re expected to be able to answer some pretty specific questions about more than just a few of them. A lot of them you’re not going to run into in simple day-to-day use cases, so for me at least, there was some study required.

Anyway, I did alright in the exam, though it’s not like there’s bonus points for anything better than passing. Out of a possible scaled score from 100-1000 points, you need at least 720. I got 956, so I’m pretty happy with that.

Aside from using AWS, I leaned on following resources for exam preparation.

• Whizlabs practice exams: I wouldn’t recommend these. The quality just isn’t there in relation to the next resource.
• DigitalCloud practice exams: Well written questions that actually represent what you’re going to see in the exam.
• https://karansingh.gitbook.io/aws-saa-c02/: This was handy as a bit of a cheat-sheet, easy reading and not overly verbose. It’s not perfect but it’s good.

1. Hit up the Crowdstrike documentation article Falcon Sensor for Mac Deployment Guide (version 6.11 and later) and that’ll have the latest correct links. Specifically, you’re going to want to get the latest MacOS Falcon agent, your Customer ID checksum and the Falcon MDM profile.
2. Head over to Meraki and after logging in, Systems Manager -> Configure -> Tags.
   • Select Add tag at the top right and select an appropriate tag type. For testing, the manual tag type may be most appropriate.
   • Set an easy name, like Crowdstrike and add a device for testing.
3. Now Systems Manager -> Manage -> Settings.
   • Select Add profile and Upload custom Apple profile.
   • Upload that Falcon MDM profile you downloaded earlier
   • Set the Scope to with ANY of the following tags and add the Crowdstrike tag to the Device tags section.
4. Next Systems Manager -> Manage -> Apps.
   • Select Add app, macOS app platform and Custom app.
   • Set the Name field to Falcon, that’s important!
   • Set an Icon URL if you feel like making it look pretty.
   • Set the Source type to Upload to the Meraki cloud before adding the Falcon agent installer.
   • You’ll want to set the Command line to sudo /Applications/Falcon.app/Contents/Resources/falconctl license $CS_CID, where $CS_ID is equal to your Customer ID checksum.
   • Set the tags again as above
   • Finally, set your own choice of other options, like Auto-install. I don’t see the point of using the Install as Managed or Remove with MDM as we’re going to want to provide a token to the agent in the event we uninstall it.
5. You’re ready now to push to your test device.

It works pretty well, at least for my needs so far.

I go digging and see there’s something about waiting for a Persistent Volume Claim (PVC). That means the storage hasn’t become available when requested. Okidoke, we can investigate the PVC and see what’s up.

Warning  ProvisioningFailed   119s                  persistentvolume-controller  (combined from similar events): Failed to provision volume with StorageClass "gp2": UnauthorizedOperation: You are not authorized to perform this operation. Encoded authorization failure message: N24SBXsb8kL_...
           status code: 403, request id: ef5acce1-...

Hm, 403? UnauthorizedOperation? That’s a bit odd. I’ve had PVCs create volumes in the past using the same credentials. What gives? Time to dig into that encoded failure message.

aws sts decode-authorization-message --encoded-message 'N24SBXsb8kL_...' --query DecodedMessage --output text | jq '.'
{
  "allowed": false,
  "explicitDeny": true,
  "matchedStatements": {
    "items": [
      {
        "statementId": "DenyUnencryptedEBSVolumes",
        "effect": "DENY",
...

Ah, that’s getting much more specific. That’s even ringing some bells. Some time ago, I had applied an AWS Organization Service Control Policy (SCP) on this account that prevented the creation of unencrypted Elastic Block Storage (EBS) volumes. In EC2 I had set up encryption by default, but the default storage provisioner in EKS was ignorant of this and was diligently headbutting the wall. I’d also noticed that it was trying to create a volume using the gp2 storage class, which has been superceded by gp3.

How to unbreak this situation? Enter the EBS Container Storage Interface (CSI) driver. I’ll spare you the nitty gritty as I was able to follow the documentation and everything basically just-worked™️. Happy days! A previously established and somewhat forgotten security rule prevented me from accidentally allowing a bad thing to happen.

The system works.

A colleague spent some time building up an environment to be able to reproduce this issue, and sure enough, there it was. He started diving in with Wireshark and all the cool tools, but couldn’t figure out what was going on. I mean, he could see what was happening, but the why was missing. So I get called in. Five minutes of googling later and I’ve fished up some relevant links. The TLDR is that Office does a pretty bad job of handling session cookies (i.e. it doesn’t), and has been handling it pretty badly for the past… decade or so.

I’m not actually here to put the boot into Microsoft. Maybe there’s a good reason for it¹. The actual point is, do you default to searching for the answer or trying to solve it yourself? Personally, it depends on the context. Is this a learning exercise or do I just need to get something done? While I might get answers to things faster, maybe my compadre ends up with a deeper, better understanding of the problem. I don’t think there’s a right or wrong approach here, just differences.

What do you think? What’s your default problem solving approach?

I read the book maybe a year or two and it’s taken me this long to get around to watching the film adaptation. A long enough period of time that my memory was getting rough around the edges on specific plot details, so take what I say ahead with a grain of salt, but…

I feel like this is a really well done adaptation. Certain scenes are basically recreated exactly as I had seen them in my mind’s eye while reading, while others have been left out entirelly. Keeping to a reasonable two hours, it’s neither long nor short and the missing scenes don’t hinder the progression at all. The cinematography is great, and combined with excellent performances from the actors, I was moved to tears more than once. The music feels of the time, but it all works.

Summary: If you haven’t read it, read it. If you haven’t watched it, watch it. 5/5 tiny birds 🐦.

So what’s been happening?

• I changed roles, which has only been good for me. If there was something greater to be learned from the my time at the previous role, I’d say that if the fundamentals feel off for more than 3-6 months, start looking. I was considering it long before following through and it wasn’t doing me any good. Several people followed me not long after; I won’t speak for them, but I suspect they were in all the same boat.
• I haven’t been smoking meats or brewing so much. I think it’s mainly the lack of space, leading to the next point…
• I’m vaguely looking at buying property, somewhere rural or semi-rural. My partner and I have relatives living in the South-West of WA and honestly, it’s a beautiful environment to be in. We both work remotely, so semi-decent internet is our driving concern. Unfortunately it’s a pretty unoriginal idea, so finding a property at a reasonable price is another matter.
• I’ve been bouldering/rock-climbing for about a year and a half now, with a few breaks for Covid-19 lockdowns. It’s a great sport, I love the community feel while still being a relatively individual endeavour. The exercise has been good for me too 😅.

I’m going to leave it there for now, but I intend to revive this a bit. We’ll see.

Don’t hold your breath.

Highest priority, I finished Black Mesa, the fan-remake of Half Life in the Source engine. Overall, I enjoyed it, but there were a few detractors.

• Performance was pretty variable, but that could be the nature of playing on Linux/Proton. Some of the biggest arenas, which should have been highlights, were a real struggle to get through with <20 FPS sometimes. Oof
• Combat overall feels more lethal than I remember the original HL
• As a corollary of the above, I tended to yo-yo between having full health and being nearly dead. The tension would have been raised if I was harder to kill but there were fewer resources around
• A similar story with ammunition; I generally wasn’t even close to running out

If you’re a fan, it’s worth a spin but for the real nostalgia kick I’d go back to HL.

Currently I’m vaporising struts in Hardspace: Shipbreaker. It’s in early access and is pretty limited at the moment, but I really like the direction and aesthetic that they’re going with.

So my last post was around Kubernetes and what the coding-on-k8s experience might be like. The TL;DR is that I’m going to wait for that to mature 😅. In general, I’m thinking about replacing the k8s setup with Docker Swarm. I had previously avoided it due to all the drama around it when it was released and bundled in with Docker; which is really a shame! It seems to provide a pretty solid experience with not nearly the complexity or pain of Kubernetes. Pair it up with Portainer and you’re starting to make containers really approachable for people that don’t want to touch a CLI. I’m not one of those people but I do work with them, so it really makes selling containerisation a lot easier.

The Swarm/K8S thing is a little apple for oranges and I’m glad that I did get some experience with K8S but let’s just say… I’m not going to be rushing back to K8S unless there’s a desperate need to use it. Which is really how it should be. ✌️

I’ve been playing around with a few tech things recently. As usual, starting things and dropping them for the next shiny.

• Buffalo: I like it but I’d like a long-form blog post on getting started. I should probably try that 😓
• Colly: scrape all the things!
• Dgraph: Pretty cool! The standalone docker image made it really easy to get going. I really ought to read a book on how best to architect things in graph databases though! Still, learn things, break things, learn more things.

Maybe expect a post on Svelte, I recently heard about this while listening to the Go Time podcast and after having a poke around the quick-start it looks pretty schmick. 👌

So like I’ve been saying, I’ve been playing with Kubernetes and have deployed it to my homelab. Maybe I’ll document that whole homelab setup one day. Anyway, in the meantime, the question is, ‘Hey, maybe I should go whole hog? Write my code and deploy to Kubernetes as well; that definitely won’t be overcomplicating things! 😅’. So off I go and install Minikube, which was surprisingly seamless. Great job to the people behind that!. Then I’m doing research on projects that make the development experience seamless, things like; I write my code and hit save, then automagically it’s running in that local minikube cluster. A few names start reoccurring, in no particular order…

• Draft
• Skaffold
• Garden
• Tilt
• Telepresence

Being lazy and not exactly an expert, Draft seemed like a decent place to start as it kept coming up in articles as being the simplest choice, and easiest to get going. I have a look around the README.md on the repo, seems alright. Cool cool, off I go to deploy. Hm, what’s the latest release. v0.16.0, hm, seems low but zero is the new black, after all. Let’s go check the date on that release. michelleN released this on 27 Aug 2018, oh, oh no. At time of writing, that’s roughly a year and a half old. I start digging into the issues.

This project is no longer being actively developed or maintained?

This is promising.

That issue was raised on 27th December, 2019; ~5 months ago. About two weeks ago, a member of the team closed the issue and updated that the project was on basically dead due to resources being allocated elsewhere. Mmhmm. As one commenter on the thread put it:

A little ‘unmaintained’ note in the readme would’ve saved me the last hour and a half trying this out. FFS.

Thankfully it was only a few minutes of my time.

So I’ve followed along with the quick start and unofficial documentation², and I hit connect on the my phone. No good, I’m not seeing any received traffic.

• I go to check the logs in the android app itself but it’s just noise, I can’t see anything useful in the context of debugging a connection. Bummer.
• I check that I’ve forwarded the correct port to the correct address on my home network. Yep, I have.
• I check that my Dynamic DNS³ is providing the correct IP. It is.
• I check that I’ve actually configured WireGuard correctly and that there aren’t any typos in the port number. Yup, all correct.

By this stage, I’m starting to get frustrated. Everyone talks about how easy WireGuard is to set up! What’s going on? I kill WireGuard and fire up nc -l <WireGuard port> on the server and nc <server address> <WireGuard port> on my desktop. The connection is made, and text entered on the client appears on the server. Alright, the port works locally. I try again using the DDNS address. Same again, it works. Okay, the forwarding on the router is working fine too. Hm. Something is different though. The DDNS address. squinting intensifies That’s a .org.

• I check the endpoint address I’ve configured in the Android app. It’s .com

So, when something isn’t connecting, check that you’ve given it the right address.

Anyway, expect some Kubernetes related posts in future.

Hidden Figures

The story of a team of female African-American mathematicians who served a vital role in NASA during the early years of the U.S. space program.

I quite enjoyed this one. There’s no single aspect that’s particularly amazing; it’s just an interesting and relatively feel-good story that’s told well. As with any story based on real life, it’s interesting to compare what’s presented against reality. In this case, the liberties taken generally don’t bother me though I do agree with the introduction of a ‘white saviour’ being problematic.

Summary: 4 out of 5 space missions.

The Invisible Man

When Cecilia’s abusive ex takes his own life and leaves her his fortune, she suspects his death was a hoax. As a series of coincidences turn lethal, Cecilia works to prove that she is being hunted by someone nobody can see.

Oof, I feel like this was a missed opportunity. There’s a strong start that falls apart when they don’t know how to progress and resolve the story. That and the writing feels lazy to me; like, the protagonist is being stalked by an invisible person and yet (mostly) doesn’t spend any time thinking about how she might render them visible. There’s another scene where the script has Cecilia basically waiting patiently in the background when she could have ended the problem; that really killed it for me. Anyway, that’s just my opinion and I’m apparently on the outside here with the film getting generally favourable reviews.

Summary: 2 out of 5 missing knives.

The Lighthouse

Two lighthouse keepers try to maintain their sanity whilst living on a remote and mysterious New England island in the 1890s.

This is one of those films where the less you know going in, the better. This film nails it on every level artistically and is as disturbing as it is excellent. Go see it.

Summary: 5 out of 5 monkey pumps.

What’s missing? Ah! Comments! What’s a blog without comments, hm? Pretty presumptuous of me to assume that anyone will be 1) reading this and 2) commenting, but I’m reassured by Cunningham’s Law. Anyway, I have a static website and I want to allow comments, but it needs to be free and I don’t want to have to run it myself.

Enter Utterances.

# utterances 🔮

A lightweight comments widget built on GitHub issues. Use GitHub issues for blog comments, wiki pages and more!

* [Open source](https://github.com/utterance). 🙌
* No tracking, no ads, always free. 📡🚫
* No lock-in. All data stored in GitHub issues. 🔓
* Styled with [Primer](http://primer.style), the css toolkit that powers GitHub. 💅
* Dark theme. 🌘
* Lightweight. Vanilla TypeScript. No font downloads, JavaScript frameworks or polyfills for evergreen browsers. 🐦🌲

If all is well when you read this post, you should be seeing it in action at the bottom of the page. The instructions provided really were quite simple and accurate, so I won’t bother with a tutorial. I did have to fork the wonderful m10c theme for Hugo to add support for Utterances, but I’ve got a pull request open so we’ll see if they bring that change in.

If someone were nitpicking they could say, ‘Liam, now you need accounts for GitLab AND GitHub; isn’t that a bit much?’. I already had both, so, no. If I wanted to keep it all on the same farm, though, I would look into Staticman. I found out about this alternative via an issue on the Utterances repo, and who knows, maybe one day down the track I’ll give it a spin.

For now, Utterances kinda works well with Indistinct Talking, don’t you think? 😉

One of my hobbies is home-brewing beer and cider; it’s something I’m still quite a novice at but there’s a great community around it and it’s something I get to enjoy the output of as well as the creative process!

It was about time that I cleaned out my spent kegs and the Fermzilla, along with the various other bits of brewing paraphernalia such as keg lines¹. Generally they look after themselves but you’ll want to flush and clean them out if you’re not constantly pouring.

Ideally you have an empty vessel that you can then fill with cleaning solutions like PBW² or sodium percarbonate. You then briefly hook up your CO2 bottle to that to give it a little pressure. Finally you hook the vessel up the keg lines and open your tap, hey presto, the cleaning solution pours through. You want to leave that in there for a little while, ~15-20 minutes, to dissolve any gunk, then repeat the whole process with water to flush the cleaner out³.

So, that empty vessel, eh? Well, I’d seen a video some time ago that looked like a handy little life-hack. Get a hand pump from Bunnings, replace the nozzle with a liquid keg post and universal poppet. You now have a vessel you can hand pump your solutions through the keg lines with, neato! All up it’s less than $20, no big deal.

So off I trundle to my LHBS, Brewmart. Pretty good shop, the staff are nice and friendly. One of the them ask me if I need any help and I tell him what I’m looking for and why. They ask if I know of the Coke bottle hack. Why no, I don’t! Turns out you can get these universal posts⁴ that screw onto Coke bottles. There’s the cleaning vessel, with one part, purchased for less than $5. Now, I actually already have one of these posts for the other common purpose, which is transferring beer from your keg to a smaller bottles. An easier way to get your home-brew to a party or function, basically.

A trip to the grocery store later and I’m now in possession of a 2L soda water bottle, very soon to be used for cleaning my keg lines. I’ll have to ponder how I forgot the line cleaning possibilities of that carbonation cap over a beer or two.

I’ll be brief, this is going to be a place for my thoughts and experiences. Probably mainly on coding, home-brewing, BBQing and gaming, but hey, we’ll see what happens.