How to Create Multi-stage Docker Builds with Harness Continuous Delivery
In the fast-paced world of software development and deployment, every minute counts. Every second is important for streamlining your processes, so you can reduce time spent waiting on build times or other lagging steps. There are a number of different techniques out there when it comes to creating container images as efficiently as possible. One such approach is using multi-stage builds with Docker, which can help reduce your container size.
When using Docker to manage your software builds and deployments, you want to make sure you are doing everything possible to simplify the process, so developers don't get bogged down with lengthy build processes. That’s why multi-stage builds are such a helpful feature of Docker.
This tutorial explains what multi-stage builds are, and how they can help speed up your development process. But first, let’s explore more of the ways you can benefit from multi-stage builds.
Benefits of Using a Multi-Stage Build
If streamlining software delivery is one of your goals, then you should definitely understand how multi-stage Docker builds work. Multi-stage builds are a great way to simplify the image creation process and save developers time.
One excellent benefit of multi-stage Docker builds is that it reduces the number of dependencies and unnecessary packages in the image, reducing the attack surface and improving security. In addition, it keeps the build clean and lean by having only the things required to run your application in production. Otherwise, developers all end up building and pushing images that are large in size with vulnerabilities that can give an easy way to attackers to get into our applications. Try using multi-stage Docker builds for optimized images and security.
Here are some other advantages of using multi-stage builds:
- Optimizes the overall size of the Docker image
- Removes the burden of creating multiple Dockerfiles for different stages
- Easy to debug a particular build stage
- Ability to use the previous stage as a new stage in the new environment
- Ability to use the cached image to make the overall process quicker
- Reduces the risk of vulnerabilities found as the image size becomes smaller with multi-stage builds
Creating a Dockerfile
Containers allow you to package up an application with all the necessary parts, such as libraries and other dependencies and ship it all out as one package. The whole application can be converted into an image and pushed to an image registry such as DockerHub. Docker is a containerization platform allowing developers to create portable, self-sufficient containers. The Docker build process starts with an image, which is only a base layer of the final image. This means that the image contains only the operating system, and any other packages needed to execute commands.
The primary purpose of Dockerfile is to create an image that can be deployed as quickly as possible and with the fewest possible dependencies. Dockerfile is a simple text document that contains all the commands and instructions to create a Docker image, which is is written as a list of instructions for Docker to follow.
The Dockerfile starts with an instruction to copy the contents of another file, called a base image, onto your computer. After this, you can add your own customizations accordingly, depending on the application you are working on. The Dockerfile is read by the Docker Engine, which then executes the instructions in order.
The next step in this process is adding layers to this base layer using layers from other images or manually installing packages.
The Dockerfile you created in the last step specifies all these steps in detail and can be used as input for the Docker build process through the Docker build command. The Docker build command is used to create an image from the Dockerfile and can be run with a tag to specify which version of the image should be created.
Creating Multi-Stage Docker Builds
Every microservice should be its own separate container. If you only use a single-stage Docker build, you’re missing out on some powerful features of the build process. In contrast, a multi-stage Docker build has many advantages over a single-stage build for deploying microservices.
A multi-stage build is a process that allows you to break the steps in building a Docker image into multiple stages. This will enable you to create images that include only the dependencies that are necessary for the desired functionality of the final application, cutting down on both time and space. With a multi-stage build, you will first build the image that contains only the dependencies needed to build your application. Then, after the image has been built, you can add in any additional layers needed to create your application and configure it for deployment. In this way, you can build images with only the code necessary for building the application. This is also strategically used to optimize the container images and make them smaller.
As mentioned above, multi-stage builds let you create optimized Docker images with only the dependencies necessary to build your application. Combined with Docker’s layered images, this can help you save significant space. The multi-stage process saves space on your Docker host and in the Docker image and speeds up the build process. In addition, the process will be much quicker than it would be if you included all the code needed to build your application.
Creating two Dockerfiles (one for development and one for production) is not ideal in the DevOps world. That’s where multi-stage Docker builds come in handy as we can have one optimized Dockerfile created for all the environments, whether it’s dev, staging or production.
Java Multi-stage Docker Build Example
To understand the concept of Multi-stage Docker builds better, let us consider a simple Java Hello World application.
Add the following code in a file named HelloWorld.java
class HelloWorld {
public static void main(String[] a) {
System.out.println("Hello world!");
}
}Then, create a Dockerfile with the following content in it,
FROM openjdk:11-jdk
COPY HelloWorld.java .
RUN javac HelloWorld.java
CMD java HelloWorld
Build the image with the following command,
docker build -t helloworld:huge .
Let’s modify our Dockerfile with the following content to show how multi-stage Docker build works.
FROM openjdk:11-jdk AS build
COPY HelloWorld.java .
RUN javac HelloWorld.java
FROM openjdk:11-jre AS run
COPY --from=build HelloWorld.class .
CMD java HelloWorld
Build the image with the following command,
docker build -t helloworld:small .
Now, let’s compare both images. Check the images created with the following command,
docker images
There is a large difference in size between the two images. This difference allows you to separate the build and runtime environments in the same Dockerfile. Use build environment as a dependency [COPY --from=build HelloWorld.class .] while creating the Dockerfile with the approach of multi-stage docker build. This will help minimize the size of Docker images.
Node.Js Multi-stage Docker Build Example
Let’s learn with a simple Node.Js application that has a basic Dockerfile.
FROM node:14-alpine
ADD . /app
WORKDIR /app
COPY package.json .
RUN npm install --production
COPY . .
EXPOSE 3002
CMD [ "node", "app.js" ]
Let’s build the image with the following command,
docker build -t [DockerHub username]/image name:tag
Push the image to Docker Hub with the command,
docker push [DockerHub username]/image name:tag
I pushed the image to DockerHub, and here is the image and size below,
Now, let’s try using the concept of multi-stage Docker build and modify our existing Dockerfile.
FROM node:14-alpine as base
ADD . /app
WORKDIR /app
COPY package.json .
RUN npm install
FROM alpine:latest
COPY --from=stage1 /app /app
WORKDIR /app
EXPOSE 3002
CMD [ "node", "app.js" ]
Let’s build and push the image with the similar commands used above. Just make sure to give a different name to the image.
Now, compare the image sizes. One with the usual Dockerfile is 48.81 MB, and the other created with a multi-stage Docker build is 7.12 MB. The image created by the multi-stage Docker build approach is more optimized and smaller.
Another example that shows how multi-stage Docker builds can be used efficiently is a scenario where you dissect the Dockerfile for different environments.
A normal Dockerfile looks like this:
FROM node:14-alpine
WORKDIR /src
COPY package.json package-lock.json /src/
RUN npm install --production
COPY . /src
EXPOSE 3000
CMD ["node", "bin/www"]
We will create three simple stages from the above Dockerfile.
- Base stage: This stage will have things in common with the original Dockerfile
- Production stage: This stage will include things useful for the production environment
- Dev stage: This stage will have components useful for the Dev environment
See the modified Dockerfile below:
FROM node:14-alpine as base
WORKDIR /src
COPY package.json package-lock.json /src/
EXPOSE 3000
FROM base as production
ENV NODE_ENV=production
RUN npm ci
COPY . /src
CMD ["node", "bin/www"]
FROM base as dev
ENV NODE_ENV=development
RUN npm install -g nodemon && npm install
COPY . /src
CMD ["nodemon", "bin/www"]
Get Started Deploying Applications with Harness
Sign up for a free trial of Harness and select the TryNextGen tab for a seamless experience.
Create a new project and select the Continuous Delivery module. Start by creating a new pipeline and add all the details that your pipeline needs.
Note: For Harness to do its magic, you need something called a ‘Delegate’ to be running on your Kubernetes cluster. Don’t worry, we have a simple tutorial to help you set up the Delegate.
Next, specify the service, infrastructure and deployment strategy for your application. Once everything is set, save the configuration and run to deploy the application.
Once the pipeline runs successfully, you should see your application deployed on the specified Kubernetes cluster. That can be verified via the kubectl command: