How template-driven CD prevents governance drift
Modern CI/CD platforms allow engineering teams to ship software faster than ever before.
Pipelines complete in minutes. Deployments that once required carefully coordinated release windows now happen dozens of times per day. Platform engineering teams have succeeded in giving developers unprecedented autonomy, enabling them to build, test, and deploy their services with remarkable speed.
Yet in highly regulated environments-especially in the financial services sector-speed alone cannot be the objective.
Control matters. Consistency matters. And perhaps most importantly, auditability matters.
In these environments, the real measure of a successful delivery platform is not only how quickly code moves through a pipeline. It is also how reliably the platform ensures that production changes are controlled, traceable, and compliant with governance standards.
Sometimes the most successful deployment pipeline is the one that never reaches production.
This is the story of how one enterprise platform team redesigned their delivery architecture to ensure that production pipelines remained governed, auditable, and secure by design.
The subtle risk in fast CI/CD platforms
A large financial institution had successfully adopted Harness for CI and CD across multiple engineering teams.
From a delivery perspective, the transformation looked extremely successful. Developers were productive, teams could create pipelines quickly, and deployments flowed smoothly through various non-production environments used for integration testing and validation. From the outside, the platform appeared healthy and efficient.
But during a platform architecture review, a deceptively simple question surfaced:
“What prevents someone from modifying a production pipeline directly?”
There had been no incidents. No production outages had been traced back to pipeline misconfiguration. No alarms had been raised by security or audit teams.
However, when the platform engineers examined the system more closely, they realized something concerning.
Production pipelines could still be modified manually.
In practice this meant governance relied largely on process discipline rather than platform enforcement. Engineers were expected to follow the right process, but the platform itself did not technically prevent deviations. In regulated industries, that is a risky place to be.
The architecture shift: separate authoring from execution
The platform team at the financial institution decided to rethink the delivery architecture entirely. Their redesign was guided by a simple but powerful principle:
Pipelines should be authored in a non-prod organization and executed in the production organization. And, if additional segregation was needed due to compliance, the team could decide to split into two separate accounts.
Authoring and experimentation should happen in a safe environment. Execution should occur in a controlled one.
Instead of creating additional tenants or separate accounts, the platform team decided to go with a dedicated non-prod organization within the same Harness account. This organization effectively acted as a staging environment for pipeline design and validation.
Architecture diagram
This separation introduced a clear lifecycle for pipeline evolution.
The non-prod organization became the staging environment where pipeline templates could be developed, tested, and refined. Engineers could experiment safely without impacting production governance.
The production organization, by contrast, became an execution environment. Pipelines there were not designed or modified freely. They were consumed from approved templates.
Guardrail #1: production pipelines must use templates
The first guardrail introduced by the platform team was straightforward but powerful.
Production pipelines must always be created from account-level templates.
Handcrafted pipelines were no longer allowed. Project-level template shortcuts were also prohibited, ensuring that governance could not be bypassed unintentionally.
This rule was enforced directly through OPA policies in Harness.
Example policy
package harness.cicd.pipeline
deny[msg] {
template_scope := input.pipeline.template.scope
template_scope != "account"
msg = "pipeline can only be created from account level pipeline template"
}
This policy ensured that production pipelines were standardized by design. Engineers could not create or modify arbitrary pipelines inside the production organization. Instead, they were required to build pipelines by selecting from approved templates that had been validated by the platform team.
As a result, production pipelines ceased to be ad-hoc configurations. They became governed platform artifacts.
Guardrail #2: governance starts in the non-prod organization
Blocking unsafe pipelines in production was only part of the solution.
The platform team realized it would be even more effective to prevent non-compliant pipelines earlier in the lifecycle.
To accomplish this, they implemented structural guardrails within the non-prod organization used for pipeline staging. Templates could not even be saved unless they satisfied specific structural requirements defined by policy.
For example, templates were required to include mandatory stages, compliance checkpoints, and evidence collection steps necessary for audit traceability.
Example policy
package harness.ci_cd
deny[msg] {
input.templates[_].stages == null
msg = "Template must have necessary stages defined"
}
deny[msg] {
some i
stages := input.templates[i].stages
stages == [Evidence_Collection]
msg = "Template must have necessary stages defined"
}
These guardrails ensured that every template contained required compliance stages such as Evidence Collection, making it impossible for teams to bypass mandatory governance steps during pipeline design.
Governance, in other words, became embedded directly into the pipeline architecture itself.
The source of truth: Git
The next question the platform team addressed was where the canonical version of pipeline templates should reside.
The answer was clear: Git must become the source of truth.
Every template intended for production usage lived inside a repository where the main branch represented the official release line.
Direct pushes to the main branch were blocked. All changes required pull requests, and pull requests themselves were subject to approval workflows that mirrored enterprise change management practices.
Governance flow
.png)
This model introduced peer review, immutable change history, and a clear traceability chain connecting pipeline changes to formal change management records.
For auditors and platform leaders alike, this was a significant improvement.
The promotion workflow
Once governance mechanisms were in place, the promotion workflow itself became predictable and repeatable.
Engineers first authored and validated templates within the non-prod organization used for pipeline staging. There they could test pipelines using real deployments in controlled non-production environments.
The typical delivery flow followed a familiar sequence:
After validation, the template definition was committed to Git through a branch and promoted through a pull request. Required approvals ensured that platform engineers, security teams, and change management authorities could review the change before it reached the release line.
Once merged into main, the approved template became available for pipelines running in the production organization. Platform administrators ensured that naming conventions and version identifiers remained consistent so that teams consuming the template could easily track its evolution.
Finally, product teams created their production pipelines simply by selecting the approved template. Any attempt to bypass the template mechanism was automatically rejected by policy enforcement
The day the model proved its value
Several months after the new architecture had been implemented, an engineer attempted to modify a deployment pipeline directly inside the production organization.
Under the previous architecture, that change would have succeeded immediately.
But now the platform rejected it. The pipeline violated the OPA rule because it was not created from an approved account-level template.
Instead of modifying the pipeline directly, the engineer followed the intended process: updating the template within the non-prod organization, submitting a pull request, obtaining the necessary approvals, merging the change to Git main, and then consuming the updated template in production.
The system had behaved exactly as intended. It prevented uncontrolled change in production.
Why this model works
The architecture introduced by the large financial institution delivered several key guarantees.
Production pipelines are standardized because they originate only from platform-approved templates. Governance is preserved because Git main serves as the official release line for pipeline definitions. Auditability improves dramatically because every pipeline change can be traced back to a pull request and associated change management approval. Finally, platform administrators retain the ability to control how templates evolve and how they are consumed in production environments.
The lesson for platform teams
Pipelines are often treated as simple automation scripts.
In reality they represent critical production infrastructure.
They define how code moves through the delivery system, how security scans are executed, how compliance evidence is collected, and ultimately how deployments reach production environments. If pipeline creation is uncontrolled, the entire delivery system becomes fragile.
The financial institution solved this problem with a remarkably simple model. Pipelines are built in the non-prod staging organization. Templates are promoted through Git governance workflows. Production pipelines consume those approved templates.
Nothing more. Nothing less.
Final takeaway
Modern CI/CD platforms have dramatically accelerated the speed of software delivery.
But in regulated environments, the true achievement lies elsewhere. It lies in building a platform where developers move quickly, security remains embedded within the delivery workflow, governance is enforced automatically, and production environments remain protected from uncontrolled change.
That is not just CI/CD. That is platform engineering done right.
