Cross build your docker images

2020-02-05 3 min read

Now that ARM is getting bigger and AWS offering compute on ARM for bargain prices I was wondering if it would be feasible to prepare most of my projects for a platform shift to ARM. The tool I like to use is buildx at the moment it is still experimental but I’m having great success with it so far. Docker Buildx is a CLI plugin that extends the docker command with the full support of the features provided by Moby BuildKit builder toolkit. It provides the same user experience as docker build with many new features like creating scoped builder instances and building against multiple nodes concurrently. And the most important thing to us it’s designed to work well with building for multiple platforms and not only for the architecture and operating system that the user invoking the build happens to run.

You can build multi-platform images using three different strategies that are supported by Buildx and Dockerfiles:

Using the QEMU emulation support in the kernel
Building on multiple native nodes using the same builder instance
Using a stage in Dockerfile to cross-compile to different architectures

We’ll focus on the first option, cross building with emulation.

docker buildx build --platform linux/amd64,linux/arm64 .

Setup

This differs from platform to platform but one thing we all have in common is pipelines, so I’ve constructed a basic buildx setup for GitLab CI/CD. If you want to crossbuild from your local machine have a look at this install guide.

Pipeline

This is a generic pipeline that takes in three variables that are pretty self explanatory:

DOCKERHUB_USER : dockerhub username
DOCKERHUB_PASS : dockerhub access token
DOCKERHUB_REPOSITORY full repository name like: aapjeisbaas/builder

build:
  image: docker:latest
  services:
  - docker:dind
  stage: build
  script:
    # install depends
    - apk add curl jq

    # enable experimental buildx features
    - export DOCKER_BUILDKIT=1
    - export DOCKER_CLI_EXPERIMENTAL=enabled

    # Download latest buildx bin from github
    - mkdir -p ~/.docker/cli-plugins/
    - BUILDX_LATEST_BIN_URI=$(curl -s -L https://github.com/docker/buildx/releases/latest | grep 'linux-amd64' | grep 'href' | sed 's/.*href="/https:\/\/github.com/g; s/amd64".*/amd64/g')
    - curl -s -L ${BUILDX_LATEST_BIN_URI} -o ~/.docker/cli-plugins/docker-buildx
    - chmod a+x ~/.docker/cli-plugins/docker-buildx

    # Get and run the latest docker/binfmt tag to use its qemu parts
    - BINFMT_IMAGE_TAG=$(curl -s https://registry.hub.docker.com/v2/repositories/docker/binfmt/tags | jq '.results | sort_by(.last_updated)[-1].name' -r)
    - docker run --rm --privileged docker/binfmt:${BINFMT_IMAGE_TAG}

    # create the multibuilder
    - docker buildx create --name multibuilder
    - docker buildx use multibuilder

    # login to a registry
    - docker login -u ${DOCKERHUB_USER} -p ${DOCKERHUB_PASS}

    # build the containers and push them to the registry then display the images
    - docker buildx build --platform linux/amd64,linux/arm64,linux/s390x,linux/386,linux/arm/v7,linux/arm/v6 -t ${DOCKERHUB_REPOSITORY}:latest . --push
    - docker buildx imagetools inspect ${DOCKERHUB_REPOSITORY}:latest

From here on out you can experiment with what platforms you can directly port to. Make sure you have a base image that supports multiarch like alpine or one of these base images.

Building for Raspberry pi

Make sure to include linux/arm/v7 for Raspberry Pi 2 and 3, older versions and pi zero use linux/arm/v6.

The Raspberry pi 4 will be able to use linux/arm/v7 and in the future if 64 bit arm becomes available might support linux/arm64

More resources

Cloud & Open-Source magician 🧙‍♂️

I try to find the KISS in complex systems and share it with the world.