Version control systems have brought in the concepts of tracking file versions, and making changes in isolation by branching, modifying, testing, and then merging to the main branch after approvals and CI checks.
The Git Experience supports the branching concept where we can follow a similar paradigm for Harness entities, such as Pipelines, Connectors, etc. We can fork out from the main branch, make changes without breaking it for others, test it in isolation, and then merge it back to the master branch – just like any other code change.
We had the following requirements for Git Branching:
To align with these requirements, we had the following challenges:
To optimize, we use Mongo as a cache, parse all of the entities, and keep them in the database. This lets us overcome the performance bottleneck that we would have faced while fetching it from Git.
Since all of the Git providers have webhooks for different events on Git, we use them to update our Mongo. Once the Git Experience is enabled, we auto-register the webhook in Git. This results in Harness receiving different kinds of events in Git.
Upon receiving events, we queue them, then asynchronously update our system. For example, when we receive a branch-create event, we update the branch list that we show in the UI. Similarly, when we receive a branch-delete event, we delete that branch from the Harness Mongo cache. Furthermore, when we receive a push-event, we fetch the content of every file in the commit and update the Harness entities.
To optimize the branch and repo filters, we store them along with other data in our entities.
Any new branch creation in Git wouldn’t always mean a change in the configuration. We offer on-demand sync in case a user wants to sync a specific branch. On triggering a sync, we pull every entity of that branch, parse them, and then store them in Mongo.
We have provided a similar user experience as git providers for users who want to add/modify entities from the Harness UI.
To identify the commits that users make to and from Harness, we use the concept of the user profile. Under the user profile, we have a source code manager where the user must provide their personal access token to make any changes to Git. An SCM is useful for auditing the person who is making changes to a Project and its Pipeline, Connectors, etc. This also helps honor the branching rules set in a repo.
While pushing to Git from the Harness UI, we have provided users with the ability to push to a new branch or to an existing branch, as well as create a PR. This will help users fork branches and get them reviewed before merging. Moreover, this will allow others to continue working on Pipelines from the main branch while the config changes are still under review.
For users to have a seamless experience in multi-repo scenarios, we list the default branch entities, and then users can filter by repo branch to get the entity. Furthermore, in the case of multiple branches, users can filter on any branch and see all of the entities, as well as apply advanced filters.
To switch seamlessly across branches and compare them, we let users switch branches while in the details section of any entity.
Git Sync and Git Branching promise a lot of potential to modern DevOps solutions. They provide an easy way to make config changes and help developers reduce the overall time to develop, test, and maintain CI/CD pipelines. In particular, branching lets all of the config changes in a branch be tested in isolation before pushing them to production. Moreover, it keeps them from blocking anyone because of incorrect config changes.
This article was written in collaboration with Abhinav Singh, Akash Nagarajan, Rama Tummala, and Deepak Patankar.
Enjoyed reading this blog post or have questions or feedback?
Share your thoughts by creating a new topic in the Harness community forum.