capsule.adrianhesketh.com

home

Go, private repositories and AWS CodeBuild

I've gotten used to the luxury of CircleCI, but I thought I'd have a go at AWS CodeBuild to bring the build pipeline inside AWS to reduce the attack surface area.

AWS Code Pipeline supports triggering builds from Github source code repositories, so I was able to get the project's source code from Github without issues. However, when you run `go get` to fetch dependencies, if any of them are private, then the build server needs to be able to authenticate against Github too as part of the build process.

At first, I created a "Deploy Key" for the project I wanted the build server to build, but Deploy Keys aren't a good match for projects which require access to multiple private repositories, so I had to setup a Github user just for the build server and setup an SSH key for it. [0]

[0]

Next, I needed to run `ssh-keygen` to create a new key, then put that into the AWS SSM parameter store so that I could get it from within a CodeBuild build step and add it to the list of SSH identities.

Once I'd added the parameter, I then needed to give the AWS-CodeBuild-Role permission to read that key.

Once that was done, I could configure the `buildspec.yml` to get the SSM parameter, add it as an environment variable, then configure the build to be able to authenticate against Github. [1]

[1]

The setup for Go on CodeBuild requires a few extra stages [2] Most important was to copy the code from the CODEBUILD_SRC_DIR into the GOPATH and then to remember to change directory into the GOPATH copy, not where the source code started out. CodeBuild lacks the feature to checkout to a specific checkout directory.

[2]

{{< gist a-h 02b883108d8322f7d4aedee55910890c >}}

Locally, I could replicate what I expected the build server to do by running Docker and then executing commands at the terminal.

However, when trying to get it to work as part of CodeBuild, I got messages in the logs from unexpected directories and the GOPATH had an extra colon after it.

Unlike CircleCI, there's no way to SSH into a broken build and attempt to fix it, so the troubleshooting workflow was tortuous - basically putting `echo` statements into some YAML and running it again.

Given that I was unable to easily replicate the problem using the same Docker container locally, I gave up after wasting a few hours fiddling with it. I might give it another try in the future.

More

Next

Exporting CloudWatch logs for analysis using AWS Athena

Previous

Adding an RBF Network

Home

home