The AI Software Factory Is a Process, Not a Product
Every team has the same coding agents. The process around them decides what ships, and it is the one part no vendor can sell you.
Every engineering team now has the same coding agents, and an AI software factory is supposed to be what you build with them. Claude Code, Cursor, Codex and GitHub Copilot are a subscription away, yet across more than 28 million CircleCI workflows, the average team's daily runs rose 59% in a year while the median team's rose 4%.
Same agents, very different results. Source: CircleCI, 2026 State of Software Delivery, more than 28 million workflows.
So when your board asks why your team does not ship like the agent-first companies, the answer is not a tool you forgot to buy. An AI software factory is the process around the agents: what gets understood before anything is generated, and who decides the output is right after it. The process is the part you build, because no vendor can sell it.
More code is not more software
After firms adopted AI coding agents, pull requests opened rose 36%. Issues resolved, which count only once the work reaches production, rose about 14%, a result the authors call borderline significant. Their abstract gives the reason in five words: "Code review becomes the bottleneck."
The study is by Fiona Chen and James Stratton at Harvard, updated in November 2026, and covers 655 firms and 445 million work events from GitHub, Jira and calendars (Artificial Intelligence in the Firm: Bottlenecks in Software Production).
Source: Chen and Stratton, Harvard, 655 firms and 445 million work events.
After agents arrived, code waited 56% longer between being submitted for review and being approved. The people who can approve a change are mostly the ones who already know the code: 83% of reviews go to engineers familiar with it, against 17% if reviewers were picked at random. Agents handed those same people about a third more to read.
Delivery pipelines show the same pattern. For CircleCI's median team, feature-branch activity rose 15% while throughput on the main branch fell 7% (2026 State of Software Delivery). The main-branch success rate fell to 70.8%, the lowest in more than five years and well below CircleCI's own 90% benchmark.
Source: CircleCI, 2026 State of Software Delivery.
Code that does merge needs more fixing afterwards. A September study of 6,774 merged pull requests from Codex, Copilot, Devin, Cursor and Claude Code found that agent changes drew a verified follow-up fix at 1.62 times the odds of human changes in the same repositories, and 69.6% of those fixes came from the same agent (Who Finishes the Job?).
The agents are producing. The teams around them are not absorbing it, which is the gap we described in AI made the coding faster, not the project, now measured across hundreds of firms.
What an AI software factory actually is
At AI Engineer NYC in October, "software factory" was the phrase that followed us from room to room. Two sessions named the problem outright: "Who verifies the software factory?" and "The Human Verification Bottleneck."
Addy Osmani's definition is the cleanest: "A software factory is harnessing loops at scale" (Software Factories, Light and Dark). In plain terms, agents run task, code and test on repeat, and something decides what ships. The real disagreement is how much agent output a person reads before it ships.
At one end is what Dan Shapiro called the dark software factory, "a black box that turns specs into software" (via Simon Willison). StrongDM's AI team runs one under two rules it publishes: "Code must not be written by humans" and "Code must not be reviewed by humans" (StrongDM).
At the other end, Osmani's light factory "moves the point of human judgment upstream, to the product, the design, and the architecture." He names the constraint plainly: "Verification, not generation, is the real constraint on a factory."
The clearest way to see the split is the inner loop and the outer loop. The inner loop is task, code, test, and agents do it well. The outer loop decides whether the output is any good: the intent behind the change, knowledge of the system, the review, and running it in production.
The factories on sale automate the inner loop: Factory's Droids, for example, take a natural-language task through planning, code and tests. The AI reviewers sold for the outer loop have not cleared the queue either: 79% of the Harvard firms had adopted one, and people still wrote most of the reviews. The outer loop is still yours to run.
The tools are portable. Context, judgment and method are not.
Everything you buy for the inner loop can be swapped next quarter: the model, the agent, the harness. Three things cannot be swapped, and they decide what the agents produce.
- The model
- The agent
- The harness
- Context
- Judgment
- Method
Context is what the agent must know about your system before it touches it: which module owns what, which rules hold, and why the billing code rounds before tax. Today most of it lives in the heads of the reviewers who already know the code, which is exactly why those reviewers became the queue.
Judgment is who decides at each gate, and on what evidence. A merge approved only because the build is green is a different decision from one where the change was also checked against the requirement. Only the second catches a change that passes every test and does the wrong thing.
Method is the order the work happens in, written down so it survives a new model, a new agent or a new hire. Without a written method, every new model and every new hire relearns your conventions from the code, one rejected pull request at a time.
If context, judgment and method live in the repo as plain files, you can swap Claude Code for Cursor and all three carry over. Lose them, and the best model available writes confident code into a system it does not understand.
How we run an AI software factory on code we did not write
Every public factory design we have read comes from a team running agents on its own product. We run ours on client code, much of it legacy and written by people who have since moved on, and we have worked this way for about a year. The client carries the risk of every merge, so the design question is where people sit in the loop, not whether a person reads every diff.
The work runs in four phases: Understand, Plan, Implement, Verify. An engineer owns each task through the first two, and the pipeline decides the last.
Models shown are our defaults. On client projects that use other models, the gates stay the same.
1. Understand. The requirement and the relevant part of the codebase are understood, in writing, before anything is generated. This is the codebase archaeology phase most AI workflows skip.
2. Plan. The better model plans: Claude Opus writes the plan, and the engineer agrees it before any code is written. Planning and building are different jobs, so a different model does each.
3. Implement. Claude Sonnet builds against the agreed plan, inside rules files that tell it what it needs to know before it edits an old codebase.
4. Verify. The session that wrote the code does not review it: a new Claude Sonnet session does, before the pull request is opened, with dedicated review skills and only the diff and the requirement, so it never sees the reasoning that produced the code. It ranks what it finds so the one real problem does not sit under forty style comments. The pull request then runs through CI/CD with unit and integration tests: a green run merges, and a failure goes to an engineer.
Claude Opus and Claude Sonnet are our defaults, not the method. We run the same gates on client projects that use other models and tools, because the phases, the rules files and the tests live in the repo, not in the model.
Legacy code gets one extra step before any of this. The Business Rule Extractor reads code nobody fully understands any more and pulls out the business rules that were never written down. Those rules become tests, so a green run on a legacy system means the old rules still hold, not only that the old tests still pass.
More gates on a review bottleneck sounds backwards. Each gate moves work earlier, so people read less at the end: the engineer agreed the plan before any code existed, the tests carry the extracted rules, and a person reads a pull request only when something fails.
Our engineers sit at the two points where judgment matters: the plan before the build, and every failure after it. That is Osmani's light factory, with human judgment moved upstream to the plan.
For your codebase, that means no change merges on the word of the session that wrote it, and every change is tested against the rules your system actually follows.
When you do not need a factory at all
Not every team needs this overhead. A small team on a young codebase it owns, with good test coverage, gets most of the gain from the tools alone, and a four-phase process would slow it down.
The process pays for itself in four situations: inherited code, more than one team on the same system, regulated work, and code that belongs to someone else. Your team may be in none of them today.
A dark factory is a real option too, for the right team. StrongDM's works for a tiny team that owns its system and built behavioural clones of the services it depends on, including Okta, Jira and Slack, so its agents can run thousands of scenarios an hour. On a legacy system nobody has that coverage on day one, which is why rule extraction comes first.
Buy the agents and keep the process light.
Build the outer loop before you add more agents, because more agents only add to the review queue the Harvard study measured.
Five questions that separate a factory from faster typing
You can run this test on your own setup this week, and it needs nothing from us.
- Are the requirement and the codebase understood, in writing, before generation? If the first artefact is code, the agent is guessing at intent.
- Is the plan reviewed before the code is? A wrong plan caught as a diff costs the whole implementation to fix.
- Does a fresh session, with none of the build context, review the code? A session that reviews its own work agrees with itself.
- On legacy code, are the business rules in tests before an agent changes anything? If not, a green build only proves the old tests still pass, which is how tests pass, CI goes green and production breaks.
- Do you measure main-branch success rate and follow-up fixes, rather than pull requests opened? In the Harvard study, pull requests opened rose 36% while issues resolved rose about 14%.
Every "no" marks a place where agent output either waits in a queue or merges unchecked. Start with the earliest "no" on the list, because each step feeds the one after it.
The part you own
The Harvard authors end with a projection to 2031. If review keeps pace with the agents, their model has software output rising more than fourfold; if review stays where it is, output rises 58% and then levels off.
Source: Chen and Stratton, Harvard, projection to 2031.
The agents are the same in both futures. What differs is how fast your team can review, and that comes from the context, judgment and method around the agents, the one part of your stack nobody can sell you.
Where to start
Every number in this post points the same way: agents add code faster than teams can confirm it is right. More agents lengthen the queue, and an AI reviewer added to the same process has not shortened it. Moving judgment earlier does: a written understanding, an agreed plan, the old rules in tests, and a person on every failure.
You can start this week without hiring anyone. Run the five questions against one live repository, take the earliest "no", and put that one gate in place before the next sprint. Each gate has a full post of its own below.
AnAr Solutions runs AI-assisted delivery on client codebases, including systems other teams built. The four phases, the review rules and the Business Rule Extractor above are how our AI-assisted development practice works.
Send us your answers to the five questions and an engineer writes back with the first gate we would add, and why.
Send your answers







