CI/CD in the AI Age: More Agents Make Integration Checks More Important

Paul Stack's three-part (1 2 3) series on CI prompted this post. I agree with much of his argument, including that agents should verify and repair their changes before opening a pull request. Paul explicitly keeps merge coordination in CI for Swamp. If you have used swamp, you know that they are developing at a fast pace.

I want to focus here on what happens when those developments land: how does the system establish that independently verified changes still work together? Who is responsible for checking that?

Imagine two changes starting from the same commit. One renames a configuration key and updates every existing caller. The other adds a new caller using the old key. Each passes its checks separately. Git can merge both without a textual conflict, but the combined code is broken. In other words, passing checks on separate branches does not establish that the combined codebase works. The example is deliberately simple. The failure only appears when the changes come together. In reality, it is often harder to catch than in this example.

The questions I want a CI system to answer are concrete: If change A lands while change B is being tested, is B's passing result valid for the state that will actually be merged or only for the application of B on their common ancestor? Can you guarantee that A+B are integrable together on top of target branch?

In a codebase with a few concurrent changes, those integration problems are manageable through people conversation and careful sequencing: People can compensate the missing integration checks by checking code and coordinating the next steps (rebasing, fixing merge conflicts, fixing changes that broke when brought together, reverting, doing more tests).

In the AI age, generating changes can be done faster.

As concurrency grows, the coordination takes more work. Manual coordination become the bottleneck.

If the integration system is not ready for this challenge, the time saved writing code can turn into time spent investigating failures, reverting changes, and coordinating repairs.

If you want to guarantee that every merge will be working together, the merge decision should depend on evidence valid for the target state. I would make the system controlling merge ordering responsible for enforcing that rule, regardless of where the tests execute.

So... which system controls merge ordering in you code base?

In general, this is the CD part of your CI/CD system.

Therefore I prefer the CI/CD system:

• Ready for multiple changes in parallel;
• Be able to handle merge ordering;
• Be able to test the ordered changed together.

For projects with many concurrent changes, I favour speculative gating and merging: testing a change with the changes expected to land ahead of it, before allowing it to merge. That targets the integration problem I care about.

Before choosing a CI tool, I would ask which state it tests: the branch alone, the branch against current main, or the speculative state of the branch (a proposed combination of queued changes). Then I would ask what happens when a queued change fails. Which results remain valid, and which combinations need testing again? That distinction matters more to this argument than the number of available workers.

This is an established integration problem. AI-generated changes make it more pressing wherever they increase concurrency. Multiple larger projects understood this in the last decades, like OpenStack (with Zuul CI) or Kubernetes (with prow).

The strongest objection is cost. Repeating every check at the merge gate can waste time without finding anything new. I agree with Paul here. My requirement is evidence for the proposed integrated state, not a fresh execution of every check regardless of relevance.

I would accept a workflow without speculative execution if it could establish that its existing verification results remain valid for the proposed combined state.

Alternatively, testing after merge can be a valid choice. This changes the failure handling: You may have already accepted a problematic integration's combination. Other changes may already depend on the broken state by the time the system detects the problem and reverts it. On top of that if contributors build on every (unvalidated) merge commit, a broken integrated state can interrupt their work. If an agent encounters that broken state while working on another change, its repair may expand the scope of the task. A change that should have been focused can become mixed with unrelated integration fixes. The change might not be as atomic as you would hope, making it harder to audit.

CI architecture depends partly on the expectations you set for your contributors on the target branch. With post-merge validation, a merge can land before the relevant checks finish. I prefer checking before merge because, in my view, handling failures on a target branch is harder than paying for the additional verification.

This also means that CI/CD behaviour largely differ in organizations because the expectations in terms of consumption/source development checkpoints are different!

When you are building your CI/CD, you need to think about your codebase's development and consumption models. Before adding more agents and increasing your rate of changes, I would check one thing: does the merge gate verify the state it is about to accept?

At TLaaS, we help teams assess what their CI/CD checks actually guarantee, where concurrent changes create gaps, and what to change. That can include implementing or maintaining Zuul or Prow.