Developer experience and CI using Docker containers

Sriram S
Technology, Engineering Productivity, Containers
Page content

In the blog titled Understanding CI/CD Personas of the series, I highlighted the importance of providing all developers with the same environment to build and test their code. This environment should also be used in the CI systems such as Jenkins for consistent software build generation. In this blog we will see how Docker containers can be used to accomplish this. I will demonstrate how I use Docker for testing my blogs and generating the static HTML files. Note: this blog assumes that you have a basic idea about Docker.

The dependencies

As mentioned earlier, I write my blogs in Markdown format and then use Hugo to generate static HTML files. These static HTML files can then be uploaded to your choice of hosting providers. I use AWS S3 buckets to host my blogs. So the hugo software is a dependency for my blog writing process. Hugo in turn supports themes that help with the presentation of the blogs. Hugo themes are typically hosted on Github. Therefore the theme code is another dependency for my blogs.

Specific version of the hugo binary is fetched using the curl command and the Hugo theme is downloaded using the git command. So curl and git are the other components I need.

Once I finish writing a blog, I generate a ZIP file containing all the static HTML files and upload that to AWS. This means I also need zip commands as a dependency.

Finally as writing and testing blogs is an iterative process, I need tools that help me setup my environment, build and test my blog site. While there are lots of technologies available, I use make which is a simple and easy-to-use tool for this purpose.

To summarize, I need the following software components and tools to effectively write, test, and generate my blog’s static web pages:

  • hugo
  • Hugo themes
  • curl
  • git
  • zip
  • make

Relating to the CI/CD Personas

Since I am a solo blog writer, I came up with the list of dependencies. In real world situations, several developers and build/tooling engineers will likely be involved. Dependency management in large software projects can be quite complicated but the gist is that all the dependencies should be discussed and listed - including their versions.

Creating Docker file with dependencies

Docker is great technology for creating consistent and repeatable environments for CI/CD. So here is the Dockerfile I use for my development and CI environment

FROM alpine:latest
 
MAINTAINER Sriram Subramanian <donotuse@invalidemail.com>

RUN apk add --no-cache curl make git zip
 
ENV HUGOVERSION 0.56.3

RUN mkdir -p /tmp/hugosrc \
   && cd /tmp/hugosrc \
   && curl -L https://github.com/gohugoio/hugo/releases/download/v${HUGOVERSION}/hugo_${HUGOVERSION}_linux-64bit.tar.gz | tar -xz \
   && mv hugo /usr/local/bin/hugo \
   && addgroup -Sg 1000 hugo \
   && adduser -SG hugo -u 1000 -h /cbblogs hugo \
 
   && mkdir /tmp/builds \
   && mkdir /tmp/themes \
   && git clone https://github.com/reachsrirams/hugo-theme-jane.git /tmp/themes/jane
 
WORKDIR /cbblogs
 
EXPOSE 1313

Let me walk you through the Docker file.

  • First the curl, make, git, and zip commands are installed in my Docker environment using the apk add command.
  • Then I initialize the HUGOVERSION environment variable. This is a crucial step because my blogs work with a specific version of hugo and the corresponding version of the theme. This dependency on a specific version is very common in large software projects as well.
  • Then the curl command is used to download, extract, copy, and setup the hugo binaries.
  • Finally the git command is used to copy the hugo themes. Note: I have forked a copy of the hugo theme from the original developer into my personal github repository.

The make and zip commands are used when the docker container is used for development.

The real benefit of Docker based developer and CI environments

I will show the usage of this docker container in the next blog but it is important to understand the benefits of using Docker based developer and CI environment.

  • Let us say, I want to allow guest bloggers to write blogs on my website. Without Docker, I would have had to communicate and help install each dependency. With Docker all these problems are solved and my potential guest bloggers will get the exact same environment and tools as I use.
  • This is very much applicable for large software projects. Not only do you want all existing developers to write and test software using the same versions, dependencies etc, but any new developers joining the team should benefit from the consistency provided by a Docker based environment.
  • As mentioned earlier, the last step of a CI system is to generate artifacts that will get tested and deployed to production. It is important that the artifact generation step is common for developers and CI systems. This will ensure that artifacts can be verified manually as well as via automation before being deployed. With the same Docker environment used for development and CI, it is possible to ensure this consistency.