Learn why leading engineering teams are managing prompts and models as AI Configs — governed, testable, and changeable without a redeploy.
Based on the ELC webinar "Beyond the Prompt: Governing AI Behavior Like You Govern Releases," presented by Harness, August 5, 2026.
A year and a half ago, most engineering teams barely talked about agentic AI. Today, 51% of large enterprises already run agentic AI in production, 78% more have plans to, and 88% of executives plan to increase AI agent budgets. The pace is the problem: a new model or prompt seems to land every week, and most teams are still shipping changes to AI behavior the way they shipped code a decade ago, through a slow, linear pipeline of edit, test, review, and redeploy that can take weeks.
The old release model wasn't built for AI behavior
In a live poll during the session, the majority of attendees said they either haven't shipped AI-powered features to production yet, or they have AI in production but changing its behavior is difficult. That's the crux of the problem: when you need to swap a prompt, a model, or a temperature setting, you shouldn't have to wait for the next scheduled deploy to find out if it works.
Treat AI behavior like a feature flag
Harness's Aaron Newcomb and Nicolás Zelaya walked through the idea at the center of the session: govern AI behavior the same way you govern a release.
Instead of hard-coding a prompt or model choice, teams can manage it as an AI Config, a governed, runtime-adjustable setting that can be changed in production in under a minute, without a redeploy.
That unlocks the same patterns teams already trust for feature releases:
- Progressive rollout: ship a new prompt or model to 1% of traffic before committing to 100%
- Experimentation: run two prompt or model variations against real users and let outcome data, not opinion, decide which one ships
- Instant kill switch — shut off a misbehaving model or prompt immediately, without rolling back an entire deployment
Governance has to travel with the change
The demo made the governance layer concrete: policy checks before a config goes live (a required Jira ticket, a mandatory environment state, a naming convention), role-based access control over who can create or promote a variation, and audit logs tracking who changed what and when. One policy example that came up directly: you cannot raise rollout exposure above 0% in production unless the same config has already been validated in a lower environment first.
The goal is to make sure nobody can ship an untested prompt straight to production without the system catching it. Good guardrails like this policy help teams move quickly and responsibly.
Experimentation closes the loop
Swapping to a cheaper model might reduce costs immediately, but panelists were candid that a cost win can mask a quality loss. By connecting AI Config exposure to real usage events, such as a support ticket's CSAT score or resolution rate, teams can see whether a change that looks good on paper actually holds up with real users, not just in an eval.
Why this matters for every team shipping AI features
The session's clearest takeaway: as access to changing AI behavior expands beyond software engineers to PMs, prompt engineers, data scientists, and the agents themselves, the old model of routing every change through a manual review and a scheduled deploy simply won't scale. Governing AI behavior the way you already govern releases is what makes it possible to move at the speed AI demands without giving up control. That means using progressive rollout, experimentation, RBAC, and having your audit trails baked in.
