Introducing Harness Infrastructure as Code Management

Uri Scheiner

Updated

Last week, during Harness {unscripted}, we announced the Beta program of a new module - “Infrastructure as Code Management.” In this blog, we will cover the background of investing in this area and why we’re so excited about it.

Background

In the last few years, we have helped hundreds of customers improve their software delivery process with modules like CI, CD, Feature Flags, and Security Testing Orchestration. These customers saw significant improvements in their release process by removing manual steps and adding intelligence to their pipelines. As we were working with these customers, we learned about the need to provide similar functionality to the Infrastructure layer.

The challenges of using Infrastructure as code at scale

Many of our customers (especially platform engineers) use infrastructure as code solutions, like Terraform, which help them manage resources efficiently and repeatedly, using git as the single source of truth. Terraform is a great tool to do that when there is a small team managing a small number of resources, but as they try to scale and increase the adoption to other teams, they will likely hit several roadblocks:

Infrastructure automation and pipeline - as part of the provisioning process, there is a need to perform additional steps (for example - code linting, security scanning, running scripts, etc.) and hook it with third parties like Jira, ServiceNow, etc. This brought up the need for an Infrastructure pipeline, which, similar to a CI/CD process, can automate and orchestrate all the needed operations before and after the provisioning process. The pipeline should be as customized as possible to support different use cases, such as running multiple steps in parallel (to speed up the provisioning process), manual intervention, automated rollback, etc.
Managing state and building security around it - You need to store your state file in a secured location, as it might contain sensitive data. Also, you need to implement a locking mechanism to avoid conflicts when various users are trying to update the same resources at the same time. In addition, you need to implement access control around it to prevent security breaches. Teams must do all that and maintain the state management, according to the company's regulations.
Identifying and handling drift - a drift between git configuration, the state file, and target environments is one of the most common challenges that we heard from customers. If, for some reason, the environment has changed without proper practice, it can lead to significant customer impact or a security breach.
Inability to estimate cost changes - many infrastructure changes may lead to excessive bills from the cloud providers. A lot of users we talked to raised that as a concern and would like to be more proactive about it and get a ballpark estimate of these changes before applying them.
Infrastructure changes can lead to misconfiguration and security breaches. The need to enforce best practices and prevent these risks was brought up by many of the users we met.

Learning all these requirements, Infrastructure as code management felt like a natural extension we should be adding to our portfolio - so we did!

How Harness IaCM addresses those challenges:

The product is in its Beta phase, and we’re inviting users to join the program and give us feedback. For now, we have decided to support Terraform as the primary IaC tool, but the plan is to go broad and provide support for additional IaC providers (more on that later in this blog).

To address the challenges our customers face, we have built the the following functionality:

Infrastructure Pipeline

Users can create advanced pipelines for infrastructure changes - you can also hook multiple plugins into the flow (like Checkov and tfsec) and run steps in parallel to expedite the execution.

Resource Visibility

‍

Users can see the resources they manage, including each resource's attributes and the Terraform-generated output.

Cost Estimation

Users can estimate how each resource's cost will change based on the new configuration.

‍

Approval Step

Users can review changes before applying them to the target environment. The approval step clearly shows estimated cost changes. At the resource level, the approval dashboard shows how many changed, deleted, and added resources there are and empowers users to inspect each resource’s attribute level changes.

‍

PR Automation

Harness will populate all the changes to git, allowing developers to review the resource change as part of the PR process.

OPA rules

Harness embeds OPA as the policy agent for its platform. Use out-of-the-box or custom written OPA rules to validate that the resources in the Terraform plan or state comply with the requirements of the organization (for example - Ensure specific AMIs are used when launching a new VM)

‍

State Management

To reduce the burden of managing and hosting the backend for State files, IaCM provides a fully managed state management, inc., the ability to see each state's revision history and compare different revisions.

‍

What benefits do Harness customers have?

Existing Harness customers have the unfair advantage of utilizing Infrastructure as Code Management with a low effort - they can leverage the same pipeline, connectors, delegates, and other Harness components currently configured to work with other modules such as CI and CD and utilize them for Infrastructure use cases. This approach reduces the friction and effort needed to onboard the new module. We already have customers that were able to start using the product within just a few minutes!