AI agents are overloading source control and CI. See how 7x growth in PRs and builds at Harness shaped Agent-Ready SCM.
The bottleneck has moved
Writing code is no longer the slow part of building software; the systems that store, review, build and test it are. Anthropic described this from the inside this month: its merges per day grew more than tenfold in a few months, and source control became its biggest development bottleneck.
Our own dogfooding data tells the same story. Over the past year, our engineering org's pull requests grew about 7x, pushes grew 6.6x, and data pulled out of repositories grew almost 6x, while the number of active contributors only grew about 30%. CI executions are also up about 7x since last year. More work done by developers and agents, now puts several times more load on source control and CI.
This blog explores what we're seeing firsthand, how coding agents are reshaping the demands on developer infrastructure, and how source control and CI tools must evolve to keep pace.
What our dogfooding data shows
By operating our own software factory, we're building and validating its capabilities under real-world conditions before bringing them to customers. Harness uses its own Software Delivery, Security Testing, Runtime Protection, and AI Cost agents alongside our Repositories and Build capabilities to develop, validate, secure, and deliver software at scale.
As AI-generated code and agentic workloads increase, we're experiencing the same challenges our customers face firsthand. Our engineering activity grew 4x to 7x across the periods measured, while the number of people committing code grew just 1.3x. This growth spans pull requests, pushes, data served, and lines of code.
In August 2026, engineering activity reached 7x its August 2025 level, measured over the full month.
More code, more pushes, far more reads
Pushes per month reached a record in July 2026, about 6.6x the same month a year earlier. Data fetched from repositories grew almost 6x, while data uploaded stayed small by comparison. That read-heavy ratio fits CI clones, automated tests and agents reading code rather than people uploading it.
Lines of code added a month grew almost 5x between March and May 2026 and stayed well above the March level through July. Over a longer stretch, active contributors grew about 30%. Code volume grew far faster than headcount, which is consistent with AI coding assistants and agents producing a large share of new code.
CI feels it too
Our internal Builds infrastructure reflects the same trend. Between November 2025 and September 2026, CI executions increased nearly 7x, while active projects grew just 1.7x. This means existing projects are running significantly more pipelines as code changes accelerate.
At peak demand, pipelines began queuing even though compute capacity was available. The bottleneck wasn't infrastructure capacity or failing jobs. It was concurrency limits struggling to keep up with a surge of parallel PR checks. Error rates remained stable, confirming that the spike reflected increased demand rather than system instability. By increasing concurrency limits and temporarily disabling high-volume, non-critical triggers, we quickly cleared the backlog.
Part of one peak was amplified by teams resuming work after a weekend environment migration. It offered a useful stress test of what happens when code generation scales faster than the systems responsible for validating it. The challenge is no longer just having enough compute. It's ensuring CI tooling can handle more frequent changes, more parallel execution, and the growing demands of AI-driven development.
Why agents change the load
Each step an agent takes lands on a system built for people. When writing code was the slowest step, source control, review, and builds had slack. When code is cheap to produce, that slack disappears.
An agent also runs the loop far more often than a person does. It pushes small changes, opens pull requests, waits for checks, reads the failures and pushes again, so one task can drive several pushes, several CI runs, and several fresh clones of the repository.
Reads grow faster than writes, almost 6x against 3 to 5x, because CI jobs and agents fetch code far more often than they change it.
What this means for Harness Code
Our source control serves reads from the server, with the Harness MCP server giving agents a direct way to read repositories and pull request data without cloning. Today that covers listing repositories, fetching files, and fetching pull requests along with their checks and comments.
The Harness CLI gives agents another way of interacting with SCM, such as harness list repository, harness get pr <repository_id>/<pr_number> and harness get commit <repository_id>/<commit_sha>.
Our experience building and operating our own software factory points to three fundamental shifts in how software development infrastructure needs to evolve.
- Source control must answer questions without a clone. As agents and CI jobs generate more activity, reading code becomes just as important as writing it. Our internal data shows reads growing faster than writes. Rather than forcing every agent to clone a repository, the Harness MCP server, Cosmos Software Factory Agent, and APIs provide direct access to diffs, merge bases, and pull request status. The Harness CLI also supports common pull request and commit operations, allowing agents to interact with source control more efficiently.
- CI needs to scale with bursts of agent-generated changes. Our CI executions grew nearly 7x in 10 months, exposing bottlenecks even when compute capacity was available. As agents create and validate changes in parallel, CI must be able to handle sudden increases in execution demand, with sufficient concurrency and capacity to keep feedback flowing.
- Code review needs to scale without becoming the next bottleneck. The volume of code added grew nearly 5x, while the number of contributors increased by only 30%. Human review alone cannot scale at the same rate. Harness AI Code Review helps close that gap by automatically reviewing pull requests, identifying potential issues, and accelerating feedback for coding agents. Internally, it has helped increase merge speed by approximately 5x, with some repositories now operating autonomously.
The common thread: tools built for human scale were not designed for this volume, and the fix is a governed software factory designed for how agents work.
What Anthropic found
We're not the only ones running into these scaling challenges. Anthropic recently shared a similar experience with its own source control infrastructure. Between May and September 2026, merged changes in its monorepo jumped from roughly 2,500 to more than 30,000 per day, a 12x increase in just four months.
Anthropic's solution was to rethink how its internal systems interacted with source control. Rather than relying on traditional Git clients, it built a custom REST API-based approach optimized for its workloads. It worked for Anthropic, but came with a tradeoff: significant engineering investment and ongoing maintenance. Anthropic itself cautioned that this approach might not make sense for everyone in their blog.
At Harness, we encountered the same underlying problem, but took a different path. Rather than replacing Git or requiring developers and agents to adopt a new way of working, we focused on scaling the source control infrastructure behind it to handle the growing demands of AI-driven development.
Scaling Harness's Agent Ready SCM
This brings us to Agent Ready SCM: In August we launched Agent Ready SCM that's built for large scale agentic usage. Given our own internal trends, our scaling efforts had started in Feb year. Anthropic built a bespoke solution for themselves but you can get 90% of the benefits using horizontally scaling Git architecture. We planned and started executing on critical scaling fronts consisting of Git cloning speed up, merge optimizations, removing bottlenecks from our Git dataplane by adding caching, development of Merge Queues for concurrent merges, improving Search API latencies and tuning database usage.
The caching work targeted clones, the reads that agents and CI repeat most. A single commit can start as many as 100 CI pipelines, and each pipeline could have several steps that begin with a fresh copy of the repository. The expensive part of serving a clone is building the packfile, and serving these identical requests over and over again. We added a short-lived clone cache keyed on the commit and the client's flags, so the first request builds the packfile and later identical requests are served the same content, without the work being repeated. This improved cloning latencies by 40%. For merge optimization, we enhanced read-only MergeVerify calls for dry-run merge checks, so mergeability checks no longer take a distributed write lock, run on every replica or go through quorum, which cut merge-verification work by about two-thirds across n replicas. We added support for Merge Queues which comes in handy in the busiest repositories with concurrent merge requests.
For CI, we scaled it by distributing load across geographies - regions and AZs, and more instance types with warm pools, and fallback strategies. We added a predictive load algorithm to perform scaling ahead of peak load given historical patterns. With this we got to serving 7x the load volumes without slowing down the build system.
We load tested our updated Repositories platform for sustained 10x current volumes and came out with following latency metrics. We can horizontally scale Code servers for higher volumes and have HPA for bursty loads enabled.
| Functionality | API Endpoint | 10x RPS volume actual P90 |
|---|---|---|
| Projects List | GET /api/v1/repos | 240 ms (api_repos_list) |
| Project Detail | GET /api/v1/repos/:id | 200 ms (api_repos_detail) |
| MR List | GET /api/v1/repos/:id/pullreq | 250 ms (api_pullreq_list) |
| MR Detail | GET /api/v1/repos/:id/pullreq/:pr | 250 ms (api_pullreq_detail) |
| MR Diffs | GET /api/v1/repos/:id/pullreq/:pr/diff | 260 ms (api_pullreq_diff) |
| Commits List | GET /api/v1/repos/:id/commits | 270 ms (api_commits_list) |
| File Content / Blame | GET /api/v1/repos/:id/content/:path | 240 ms (api_content_file) / 270 ms (api_blame) |
| Code Search | GET /api/v1/search | 810 ms (api_search) |
| Git Clone | GET /info/refs?service=git-upload-pack + POST /git-upload-pack | 890 ms (git_clone_info_refs (390 ms) + git_clone_upload_pack (770 ms)) |
| Git Push | GET /info/refs?service=git-receive-pack + POST /git-receive-pack | 360 ms (git_push_info_refs (190 ms) + git_push_receive_pack (170 ms)) |
| Git Pull | Same as clone, incremental pack | N/A (Not explicitly separated in dataset) |
| Git LS-Remote | GET /info/refs?service=git-upload-pack | 390 ms (git_clone_info_refs) |
What's coming next
We have not stopped here. With agents' code volumes rising, we know that we will need to scale our SCM even further. This is where we've started working towards an architecture where compute and storage are separated. This shift is not a new problem in distributed systems; rather, it represents a well-established pattern that has fundamentally reshaped modern data infrastructure. Several mainstream databases and data warehouses such as Snowflake, Amazon Aurora and Redshift successfully operate on this principle. Idea is to decouple processing layers from underlying storage media to allow both to scale independently and cost-effectively.
This brings us to what's coming next for Agent Native Source Code Management:
We are working on changes below the surface of Harness Repositories: moving where code repo data is kept, without changing how developers work. Durable cloud object storage becomes the home of the repository. The git servers that handle requests keep a local copy for speed, and that copy can be created, replaced or removed at any time.
The workflow, at a high level:
- Developers and Agents push and pull as they do today. Nothing changes in the IDE, the CLI or git itself.
- Harness Code continues to run pull requests, code owners and merge rules.
- Behind it, the git servers act as fast, disposable caches over the durable store, instead of each holding the only full copy.
- Hosted pipeline runners may in future fetch repository data from the same durable store, which would take clone traffic off the git servers.
This will further improve write throughputs on busiest repositories, lower the pressure of cloning, increase read/write capacity by horizontally scaling the caches and ease disaster recovery.
Conclusion
Earlier this year, we introduced our new Agent-Ready SCM, bringing capabilities to Harness Repositories designed to support source control for code at machine speeds. Faster Git cloning, optimized merges, caching in the Git data plane, and server-side reads help handle growing workloads, while the Harness MCP server gives agents direct access to repository and pull request data without requiring a local clone. This architecture continues to evolve further with separation of compute from storage.
These improvements were driven by the scaling challenges we experienced firsthand in our own software factory and engineering teams. By addressing these bottlenecks, we're enabling teams to delegate more work to agents without source control slowing them down. And as AI-driven development continues to accelerate, we're evolving our platform to keep pace with the growing demands of developers and agents.
