We're excited to announce the general availability of Cache Intelligence to simplify the process of implementing caching in continuous integration (CI) pipelines by automatically caching and restoring known dependencies. With the release of this feature to the Harness Continuous Integration module, developers can optimize CI build times and improve the efficiency of software delivery.
CI builds often require downloading and installing large dependent software components, such as libraries, which can be very time-consuming. When building locally or reusing build machines for multiple builds, a cache is usually available to speed up the process. However, this is often not the case when using modern continuous integration solutions because they execute pipelines in ephemeral environments, such as containers or virtual machines, which are terminated after execution. This approach ensures that pipelines run in isolated environments, free from interference by other build instances, but the lack of a local cache can result in longer build times. As such, configuring a cache is recommended when using ephemeral build environments, but this task can be tedious.
The responsibility of incorporating this logic into pipelines, and managing the cache typically falls on the user. Even though caching can already be achieved in various ways — such as save and restore cache steps or mounting volumes — engineers are still responsible for the bulk of configuration and maintenance, taking time away from development. Many developers don’t even realize that caching is a solution to expedite their build times.
Harness Cache Intelligence simplifies the build process in CI pipelines by streamlining cache management and automatically caching and restoring known dependencies. By removing the complexity of cache management, developers benefit from more streamlined workflows, allowing them to increase efficiency and productivity and focus on writing code.
Let’s demonstrate how users can get started with Cache Intelligence to see how this new feature significantly reduces build time.
In order to show how effective Cache Intelligence is in reducing build time, we’ve used the Harness Developer Hub (a web application built with Yarn) to conduct two consecutive builds.
In the first execution, the cache was unavailable, and the build step took 9m 5s, with the entire build process totaling 10m 56s.
In the second execution, the cache was restored and utilized, reducing the build step to 3m 10s and the overall build time to just 6m 1s.
The result is a 45% reduction in the total build duration, demonstrating the significant impact of Cache Intelligence on optimizing build performance.
In the images below, you can see the difference in performance between individual steps when the cache is not present (the first is the first build) and when the cache is available (the second image is the second build). This comparison showcases the significant improvements in build speed and efficiency from Cache Intelligence.
When using Harness CI, you first need to add the “caching” property in your CI stage. We store the cache securely in Harness cloud, our fully hosted environment, freeing you from the hassle of providing your own storage.
Currently, automatic dependency detection and caching supports Bazel, Maven, Gradle, Node, Yarn, and Go build tools. If you're using a different build tool or have dependencies stored in non-standard locations, Cache Intelligence can still be used by specifying the paths you'd like to cache. For instance:
Here’s a step-by-step example demonstrating how can you use Cache Intelligence:
After following these steps, you will have a basic ‘hello world’ pipeline that clones your forked repository and prints a welcome message. Modify the pipeline as following
Add the ‘caching’ property under the stage’s spec property
Replace the ‘Echo Welcome Message’ step with the following step that builds the application
At the end you will have a pipeline similar to the following:
Harness' Cache Intelligence presents an advanced approach to enhancing build performance in CI pipelines. Ready to start streamlining your cache management? Sign up for a free trial today, or request a demo!
Enjoyed reading this blog post or have questions or feedback?
Share your thoughts by creating a new topic in the Harness community forum.