The Power of Container-Based CI

This article was written in collaboration between Uri Scheiner and Dinesh Garg.

Traditional CI tools (like Jenkins, for example) are designed in a way that forces the user to make sure that the clients (from which the steps are being executed) contain all the libraries that are needed for the step to run. For example: if the step is executing an HTTP request, an HTTP client must be installed. If the step is deployed to a K8s cluster, a kubectl is needed, etc. 

This architecture is difficult to maintain. A large implementation of CI can easily get out of control for several reasons:

1. Users have to make sure all clients (or ‘slaves’) are up to date. If there is a new version of the CLI you want to use, you’re on the hook to roll the new version out to all clients.
2. You might be interested in using different versions of CLIs (for example, different versions of Terraform) against different machines. But since you can’t install different versions of the same CLI in the same machine, you need to have different machines, for different versions, and you have to make sure you’re choosing the right one for every execution, otherwise, the build will fail... 

This is a nightmare for every CI team, which leads to a lot of time wasted, failure, and mainly frustration.

Fortunately, Harness CI is designed differently. Harness CI is container-based, which really takes away all the burden of managing libraries in slaves and gives you full control and visibility of which libraries and CLI are needed in every execution. 

How Does it Work?

Let’s see what happens when you author a CI pipeline in Harness.

After you add a build stage to your pipeline, you can start adding the steps that will get executed when the build is triggered. 

For each step, there are only a few things you need to define:

1. Step name (that will show up in the pipeline).
2. The image to use when launching the container (you should have all libraries baked in).
3. The commands to run, once the container is up. 

And you’re ready to go!

When the pipeline gets executed, Harness manages the complete lifecycle of each of these steps:

• Spinning off a container on the infrastructure of your choice. 
• Running all the commands.
• Storing logs and populating the results to the UI or the HTTP response.
• Tearing down the container at the end of the execution.

As the containers are ephemeral and immutable, you don’t have to worry about updating the libraries. If you want to use a new version of a CLI, just set the step to use a different image. Interested in using different versions of the same library? Simply specify the tag of the image to pull at runtime. 

In order to save on resource consumption, Harness seamlessly determines the optimal pod specification based on your pipeline configuration. For example, in the screenshot below, the memory and CPU allocated would be done as follows:

Max (S1, S2, {S3+S4+S5})

Max (2GB & 1CPU, 3GB & 1CPU, {(4GB & 1 CPU) + (2GB & 1 CPU) + (2GB & 1 CPU)})

Max (2GB & 1CPU, 3GB & 1CPU, 8GB & 3 CPU) -> 8GB & 3CPU

What Are the Pros of Using a Container-Based CI Product?

You should use a container-based CI solution if:

1. Containers are approved in your organization.
2. You are familiar with docker images, know how to create images that contain the libraries you need, or have access to them.
3. You want to work with a product that is super easy to use and extend using plugins.  

Conclusion

In this article, we learned about container-based CI tools and how they differ from a traditional tool like Jenkins. The world truly is heading towards container-based architecture - just look at the bulk of CI/CD tools out on the market now. Most are cloud-native, and old tools like Jenkins (at over a decade old) simply aren’t the way anymore. We highly encourage you to try Harness for yourself and see how simple CI can be. 

If you’d like to do some further reading on CI, why not read about Test Intelligence? This new feature is designed to save you shiploads of time - move fast, without breaking things.