AI coding is changing the economics of software production. What used to require days of engineering time can now become a runnable interface, a repository-level patch, or a first test suite in a few hours. Coding agents can read a repository, plan a task, edit files, execute commands, and iterate against the feedback they receive.

One consequence is easy to miss: code is becoming cheaper, while trusted delivery remains expensive.

A demo that runs and a system that can be merged, released, and maintained with confidence are still different things. The second requires answers to harder questions:

• Does the change satisfy the request, rather than merely pass a local test?
• Did it preserve permissions, compatibility, performance, and existing boundaries?
• Is the available evidence sufficient, and what else should be checked?
• Did the fix solve the original problem without creating a new regression?
• When the evidence is incomplete, should the system repair itself, block delivery, or ask for human review?

The destination of a Software Factory should not be “an agent generated code.” It should be “the system can continuously produce deliverables that people can trust.”

That requires two kinds of model capability: generation and verification.

A software factory needs quality control, not only a production line

A factory does not remove inspection, sampling, process control, or release procedures because it has faster machines. A Software Factory should be designed the same way.

The generation model turns a request into a candidate implementation. It analyzes the task, retrieves context, proposes an approach, changes code, calls tools, writes tests, and tries to complete the work. Its objective is to increase the probability that something useful gets made.

But a candidate implementation is not trusted delivery. The verification model decides whether there is enough evidence to release it, identifies missing evidence, chooses the next check, and sends structured feedback to the generation model. Its objective is not to produce more code. It is to increase the probability that the result can be trusted.

“Two models” does not mean every system must deploy two completely separate sets of weights. It describes two functions and responsibilities that should not be collapsed:

Dimension	Generation model	Verification model
Core task	Generate candidate code and an execution plan	Decide whether the result is trustworthy, how to prove it, and how to repair it
Optimization target	Task completion and implementation speed	Risk calibration, evidence sufficiency, and deliverability
Main inputs	Requirements, repository, tool feedback, and prior context	Candidate change plus tests, CI, logs, defects, runtime results, and policy
Main outputs	Code, configuration, commands, tests, and patches	Ship / Block / Human Review, an evidence chain, a verification plan, and repair guidance
Key data	Code, tasks, and execution traces	Verification Trace: evidence, defects, repairs, regressions, and the final decision

The shortest version is:

The generation model asks, “Can we make it?” The verification model asks, “Can we trust it enough to deliver it?”

A stronger generation model does not automatically create trusted delivery

As models improve, it is natural to assume that a sufficiently capable generation model will eventually write, test, review, and approve its own work.

That is a useful engineering intuition, but it is not enough for production reliability.

First, a generation model works inside a candidate space. It has to propose a useful path with incomplete requirements, limited context, and a finite execution budget. Even when a patch passes the available tests, it may not have covered maintainability, ownership boundaries, migration cost, backward compatibility, production behavior, or the future debugging burden.

Software quality is also not a single metric. A change can be functionally correct but wrong about permissions. Tests can pass while compatibility is broken. Local performance can improve while observability gets worse. A fix can remove the current defect and create a later maintenance problem. A generation model may attempt verification, but its own confidence cannot substitute for independent evidence.

This is why “the task is complete” and “the problem is actually solved” must remain separate claims. A green test suite is evidence. It is not automatically a complete account of the real software problem.

A stronger generation model will improve the quality of candidates. It will not remove the need for verification. In fact, the longer the task and the more complex the system it can change, the more valuable an independent verification layer becomes.

A harness is necessary, but a harness is not a verification model

A harness is the infrastructure that lets AI coding become useful beyond a demo. It provides tools, a sandbox, code execution, tests, retrieval, version control, observability, and permission boundaries. Without a harness, a model can only say that it wrote the code correctly. With one, it can run the code, execute checks, observe logs, reproduce a failure, and receive external feedback.

But a harness produces and executes evidence. It does not automatically decide what the evidence means.

A passing test does not by itself prove that the requirement is satisfied. A clean static scan does not prove that there is no security risk. A green CI pipeline does not prove that the intended revision was actually tested. Higher coverage does not prove that the important business risk was covered.

The verification model should decide:

1. Which evidence the task needs most;
2. Whether the available evidence is sufficient;
3. Whether different signals contradict one another;
4. Whether the risk has crossed a blocking threshold;
5. Which check to run next, or how the generation model should repair the change.

The more accurate architecture is:

The generation model proposes a candidate
        ↓
The harness runs tests, scans, simulations, and observations
        ↓
The verification model aggregates evidence, judges risk, and chooses the next step
        ↓
The generation model repairs or continues verification
        ↓
The system records an auditable delivery decision

The harness is the execution layer of verification. The verification model is the intelligent layer that controls quality and release.

Self-checking is useful, but it is not independent verification

There is no need to reject same-model self-checking. Asking a generation model to inspect its diff, add tests, and revise its work can improve a first draft. It should be the first line of defense in an AI coding workflow.

But “self-checking is useful” does not mean “self-checking can own production release.”

When generation and judgment depend on the same model, the same context, and the same early assumptions, they can share the same blind spots. The model may misunderstand a permission rule, miss an undocumented business constraint, or fail to notice a dependency change. Calling the same model again does not automatically create new runtime evidence or an external ground truth.

The right relationship is:

Self-checking improves generation quality; independent verification carries release responsibility.

The generation model can inspect its own work. The verification model must still be able to use external evidence to reject, block, or correct its conclusion.

What does a verification model do?

A verification model is not merely a model that writes better review comments, and it is not a wrapper around static rules. It has at least four responsibilities.

1. Understand the delivery contract

The object of verification is not an isolated code fragment. It is the relationship between the requirement, the implementation, and the constraints around it.

The verification model needs to understand what the user asked for, how the existing system works, which boundaries the change touches, which behavior must not break, and what evidence could prove that the goal was reached.

2. Choose a verification strategy

Not every change needs the entire test suite, and not every risk can be found by unit tests.

The verification model should decide whether the change needs checks for permissions, state consistency, API compatibility, dependencies, performance, migration behavior, concurrency, recovery, or the user flow. When evidence is missing, it should produce an executable plan and invoke the harness.

3. Aggregate heterogeneous evidence

Diffs, test results, CI output, static scans, runtime logs, alerts, issues, repair history, and user feedback are all incomplete on their own.

The verification model organizes these signals into an evidence chain and asks whether they support or contradict the claim that the current artifact is deliverable.

4. Make a decision and guide repair

The output should not stop at “there may be a problem.” It should include:

• Ship: the evidence is sufficient to merge or release;
• Block: an unacceptable risk exists, with the evidence that supports the block;
• Human Review: the evidence is incomplete or the business risk is high;
• Repair Guidance: what the generation model should change, test, and verify again.

That is the difference between a verification model and an ordinary review product. A review product mainly produces suggestions. A verification model produces an explainable delivery decision.

From code data to a Verification Trace

Generation models learn from code, task descriptions, examples, and tool-use traces how to complete a task.

A verification model needs another distribution of data: evidence about why an artifact was trusted or rejected.

We call this a Verification Trace. A complete trace includes:

Task and constraints
→ Candidate code, configuration, and agent execution
→ Tests, CI, scans, logs, and monitoring evidence
→ Defect discovery and diagnosis
→ Repair attempts
→ Regression and re-verification results
→ Ship / Block / Human Review decision
→ The real result after delivery

This is different from “more code data.” Code tells a model what people wrote. A Verification Trace tells it which change was ultimately shown to be reliable, which change failed, where the evidence was, and how the system should correct itself.

Public issues, pull requests, CI fail/pass pairs, and repair histories can provide cold-start examples. Synthetic and active scenario construction can fill in long-tail cases such as permissions, payments, concurrency, state consistency, migrations, and regressions. Real user and enterprise environments can form private trace graphs without moving raw data outside the customer's boundary.

The principle is simple:

Customer data stays in its domain; verification knowledge can transfer.

The center can learn authorized abstractions, synthetic cases, aggregate signals, and general verification methods. The enterprise can retain private policy, code, logs, and complete evidence.

How verification improves generation

The value of a verification model is not limited to the final “reject.”

A real Software Factory forms a loop:

Generate
→ Execute
→ Observe
→ Verify
→ Intervene
→ Repair
→ Re-verify
→ Learn

When the verification model finds drift from the objective, redundant exploration, missing tests, an ineffective repair, or a likely regression, it can:

• identify the missing verification step;
• add or generate a targeted test;
• stop an unverified change from being submitted;
• return the agent to a requirement it overlooked;
• provide an evidence-backed repair path;
• preserve the process as training data for the next cycle.

The verification model is therefore an external control layer and a high-quality feedback source for the generation model.

It can also remain neutral across generation systems. Whether the candidate comes from Claude Code, Codex, Cursor, Qwen, Kimi, or an internal agent, the verification model should make a consistent quality decision from evidence rather than endorsing one generation ecosystem.

Why this is a window for verification models

The opportunity exists not because AI coding is weak, but because it is becoming strong and common.

When AI only completes a few lines, human review can absorb most of the risk. When agents start handling repository-level repairs, feature work, cross-file changes, and automated release, human review becomes the throughput bottleneck. Quality risk grows from a single function into a system-level delivery problem.

The market is already allocating budget to quality, review, and governance after generation. That shows demand for a layer beyond code creation, but it does not mean the verification problem has been solved. The open question is who can provide an independent, auditable, continuously improving delivery decision across models, tools, CI/CD systems, and private enterprise workflows.

That is the opportunity for verification models.

What should a verification model be tested on?

“Verification model” cannot remain a persuasive phrase. It must prove itself with a stricter standard than generation models.

At minimum, it should be tested on:

1. Defect discovery: Can it find functional, regression, security, compatibility, state, and maintainability risks?
2. Evidence sufficiency: Can it identify cases where tests pass but the evidence is still inadequate?
3. Risk calibration: Does it block high-risk problems while avoiding excessive false positives on low-risk changes?
4. Repair guidance: Does it help the generation model reach a trusted repair faster and with less rework?
5. Long-term delivery: Does it reduce production defects, rollbacks, repeated repairs, and human review burden?

This is why new benchmarks matter. Traditional software benchmarks mostly ask whether an issue was fixed. A Software Factory also needs to measure trusted delivery across new features, maintenance, regression verification, and long-term evolution.

The prize is not a single leaderboard score. It is a better standard for deciding when AI-generated software can be trusted.

Conclusion: the next layer of software factories is verification

The first half of AI coding competes on who can generate software faster and more capably.

The next half will compete on who can turn a large supply of candidate code into reliable, auditable, continuously evolving delivery.

A Software Factory cannot have only a production line. It also needs quality control.

The generation model makes the candidate.
The harness runs the work and collects evidence.
The verification model understands the evidence, judges the risk, guides repair, and decides what can be trusted.

Generation makes software supply grow quickly. Verification decides how much of that supply can enter production.

Valkor is interested in this layer of AI-native software infrastructure: not replacing generation models, but giving the software they produce a more trustworthy, verifiable, and deliverable path into real systems.

References

1. Lilian Weng. Harness Engineering for Self-Improvement, 2026.
2. Yang et al. Are “Solved Issues” in SWE-bench Really Solved Correctly? An Empirical Study, 2025.
3. Huang et al. Large Language Models Cannot Self-Correct Reasoning Yet, ICLR 2024.
4. SWE-PRBench, 2026.
5. GitHub Code Quality billing, 2026.
6. Anthropic. Claude Code Review, 2026.