
In this article
A successful build is one check. It shows that the changed code compiles and packages, and it says nothing about whether the workflows the business relies on still behave as they did. In a peer-reviewed study of 30,548 automatic dependency updates in Java projects, 2.30% had behavioral breaking changes that impacted client tests (Understanding the Impact of APIs Behavioral Breaking Changes on Client Applications, ACM FSE 2024); the figure counts only the changes that existing tests caught and concerns library updates, not changes to application code. In application migration testing for one application, acceptance is agreed before the code changes, while the original behavior of the unchanged version can still be observed and recorded as a baseline.
This article covers a change to one existing application after the decision to change it has been made; moving infrastructure to another environment and reconciling migrated data are outside its subject.
Start acceptance before the code changes
Acceptance starts while the unchanged version of the application still runs, because that version is the most direct source of expected results.
The ISTQB Glossary gives the vocabulary. A test oracle is "A source to determine an expected result." An expected result is "The observable predicted behavior of a test item under specified conditions based on its test basis." For a change to an existing application, the unchanged version works as a test oracle: given the same inputs, its outputs are the expected results for the changed version, next to the requirements.
AWS describes the same construction step by step in its Prescriptive Guidance Replatforming mainframe applications using a shared IBM Db2 for z/OS database for gradual migration, Testing section. The guide says: "Start by identifying the critical business processes and transactions that need to be compared between systems." Test data is made "identical for both systems to enable accurate comparison." The full suite runs on the original application first: "Record all transactions, inputs, and outputs." Then the target follows: "Run the same suite of tests on the replatformed application. Make sure that all target system tests use the same inputs as the source system tests." The last step is to "Document all test results, discrepancies, and performance metrics." The guide describes moving a COBOL application off the mainframe. The reader of this article works with Java, .NET or PHP, so the construction carries over as a method, and the guide says nothing about results on those stacks.
Once the code is replaced, the original behavior is no longer observable, and expected results have to be rebuilt from memory and documentation.
The recording follows the construction in the AWS guide. For each named workflow, a fixed set of inputs runs on the unchanged version, and what comes out is stored next to the inputs: returned values, screens or responses, files that a batch job writes and, where the workflow writes to a database, snapshots of the affected tables taken before and after the run. The database returns to the first snapshot before the changed version runs, so that both versions start from the same state. The recording breaks down where the output depends on something the record does not hold: the current date and time, generated identifiers, calls to external systems that the agreed environment lacks, or data that differs between runs. Such inputs are pinned, or the fields they affect are named as acceptable differences before recording. A workflow that no prepared set of inputs reaches stays unrecorded, and the record says so.
Limit of the method. The method needs an original behavior that someone can observe. When the application has no reproducible build, no environment in which the unchanged version runs, or behavior that is neither recorded anywhere nor checkable, there is nothing to compare the changed version against, and the method does not start. In that situation the first step in DFG's method is a separate paid preparation, or no work. When the unchanged version cannot be observed for a workflow, expected results can only be reconstructed from documentation and from the people who use and maintain the application, and the record marks each such result as reconstructed, not observed. A reconstructed result is weaker evidence: it shows agreement with a description of the behavior, not with the behavior itself, and the acceptance decision for that workflow carries the same limit.
The six parts of an acceptance record
An acceptance record is one document with six parts, agreed before the change by the application owner and the QA lead, and accepted after the checks by the business process owner.
| Part | What the record holds |
|---|---|
| Important workflows | The business workflows the change must keep working, named one by one |
| Expected results | For each workflow and set of inputs, the result the changed version must produce, taken from the unchanged version or from the requirements |
| Acceptable differences | Each difference in output allowed in advance, and who allowed it |
| Old defects versus new regressions | Each known old defect, recorded before the change, so that a failure after the change can be classified |
| Agreed data and environment | The data and the environment the checks use, approved by the owner of data access |
| Reproducible checks | How the check is repeated: the same inputs, the same environment and the same steps on both versions |
| Who signs | The application owner and the QA lead agree the record before the change; the business process owner accepts the result for each workflow. This row is the sign-off under the record, and the record still has six parts |
Naming the workflows comes first. Microsoft's Cloud Adoption Framework page Prepare workloads for cloud migration to Azure says: "Test all critical user workflows and business processes". That page is about moving a workload to the cloud, a subject outside this article, and is cited here only for naming the workflows to test. Which part of the code to change first is a separate decision, outside this article.
An expected result holds only "under specified conditions", in the ISTQB wording, so the record states the conditions next to each result: the inputs, the data and the environment. An acceptable difference is a decision written down before the checks run: which part of the output may differ between the two versions, and who allowed it. Each such rule narrows the check on purpose. A difference that changes what the workflow delivers is a decision for the business owner of that workflow. Which differences are acceptable depends on the application and the stack, so the record settles them in advance instead of leaving them to be argued after a failed check.
The owner of data access decides which data and environments the checks may use, and the record names that decision. A recognized wording of the practice comes from NIST SP 800-53 Rev. 5, Release 5.2.0, controls SA-3(2) and PM-25: approve, document and control the use of live data in "preproduction environments" (development, test and integration) and use "test or dummy data" where possible; it is a catalog of controls written for US federal systems, cited here as a wording of the practice and not as a claim of compliance. For the purposes of this method, DFG works on test data in an environment agreed with the client, and the record names both.
The environment has to be complete: the Microsoft page warns that "Missing components can lead to false positives or undetected issues during testing." The AWS guide adds an isolation rule for the comparison itself: "Make sure that the data table isn't being used by other applications during the test, because this can affect the equivalence test."
For the operations side, ISTQB names operational acceptance testing: "A type of acceptance testing performed to determine whether the organization responsible for operating the system can accept it." In the agreed environment, the checks for the operations side cover the batch windows the application has to fit, the logs it writes, monitoring, the start after a restart and the rollback.
A check that passes and fails on the same code cannot tell a regression from noise, which is why the record has a field for how each check is repeated. ISTQB has a name for comparing two versions on the same inputs: back-to-back testing is "A test approach in which a pseudo-oracle is used." A pseudo-oracle is "An independently derived variant of the test item used to generate results, which are compared with the results of the original test item based on the same test inputs."
For high-risk workflows, parallel run testing is an option: both versions receive the same requests and the outputs are compared, which works only for workflows that read data or whose writes are suppressed, because issuing a policy, taking a payment or sending a customer letter cannot happen twice.
The same record applies when a migration tool produced the change: the tool's result is checked against the original behavior, and this article describes how. A separate article covers what a migration tool leaves.
Old defects and new regressions
A failure after the change is a known old defect when it reproduces on the unchanged version with the same inputs in the same environment, and a new regression when it does not. ISTQB defines regression testing as "A type of change-related testing to detect whether defects have been introduced or uncovered in unchanged areas of the software." The definition covers both cases, so the record lists known old defects before the change.
| Known old defect | New regression |
|---|---|
| Reproduces on the unchanged version with the same inputs in the same environment | Does not reproduce on the unchanged version with the same inputs in the same environment, and reproduces on the changed one |
| Enters the record before the change | Enters the record after the checks on the changed version |
A third case sits between the columns: an old defect that the change made visible. By the ISTQB definition it is a finding of regression testing, because the defect was uncovered, and by origin it is old. The application owner decides who fixes it, because the defect belongs to the original code, and the fix is a separate item outside the change unless the record includes it. The defect enters the record as a known old defect with a note that the change made it visible, so it does not count as a new regression of the change. When the roles disagree on whether a case is an old defect or a regression, the QA lead reruns the check on the unchanged version, and where that version no longer runs, the business process owner of the workflow decides and the decision is written into the record. Without the unchanged version and its recorded outputs, telling an old defect from a new regression comes down to opinion.
An illustrative acceptance record
Synthetic example: no client data.
The record below is filled in for a premium-calculation workflow in an internal application of an insurer. It shows the shape of the entries and carries no numbers or rating factors.
| Field | Entry |
|---|---|
| Workflow | Premium calculation for a fixed set of policy inputs in the internal application |
| Expected result | The premium the unchanged version returns for the same inputs |
| Acceptable difference | The timestamp format in the calculation log |
| Known old defect | A defect recorded before the change and reproduced on the unchanged version |
| Data and environment | Synthetic data, or data the owner of data access has approved, in the agreed test environment |
| How the check is repeated | The same set of inputs run on both versions in the same environment, with the outputs compared |
| Who signs | The application owner and the QA lead agree the record before the change; the business process owner of the premium-calculation workflow accepts the result |
A difference in premium rounding between the two versions is a decision for the business owner of the workflow.
Acceptance is not release
Acceptance and release are two separate decisions, taken at different times on different grounds.
Acceptance decision. Whether the result is accepted against the record. The grounds are the regression evidence and the known differences for each named workflow.
Release decision. When and how the change goes to production. The grounds are a release and rollback plan designed for the particular system.
DFG's method keeps the two apart. A bounded code change is accepted against named workflows, and the client receives code, build instructions, regression evidence, known differences and a handover. It is not a release permit. Release, data migration, further applications and support are separate purchases, and accepting one change does not oblige the client to upgrade the next application.

Agree the record before the change
A successful build shows that the code compiles; acceptance shows that the named workflows still produce the agreed results. The record that makes this possible is written while the old version still runs, separates known old defects from new regressions, and leaves the release decision to a later, separate step.
Two agreements happen at different times. Before the change, the application owner and the QA lead agree the record. After the checks, the business process owner of each workflow accepts the result for that workflow, in line with the Microsoft guidance to "Conduct acceptance testing with business users to confirm that the workload meets business expectations and user experience requirements." The owner of data access approves the data and the environment.
The next step is to agree the acceptance record with the application owner and QA lead before committing to the change.
Frequently asked questions
Is a successful build enough to accept an application change?
No. A successful build shows that the changed code compiles and packages. Acceptance asks whether the named workflows still produce the expected results under agreed conditions. ISTQB defines regression testing as testing "to detect whether defects have been introduced or uncovered in unchanged areas of the software," and a build does not look for such defects.
When should acceptance criteria be written?
Before the code changes, while the unchanged version still runs. Its outputs on recorded inputs can then serve as expected results, and old defects can be recorded as known. After the change, the original behavior is no longer observable. The application owner and the QA lead agree the record before the work starts.
Which workflows should be in the acceptance record?
The business workflows the change must keep working, named one by one, starting with the critical ones. AWS guidance on replatforming starts by identifying the critical business processes to compare, and Microsoft guidance asks to test all critical user workflows. Each workflow gets expected results, acceptable differences and a repeatable check.
How do we separate old defects from new regressions?
Run the failing check on the unchanged version with the same inputs in the same environment. If the failure reproduces, it is a known old defect, recorded before the change. If not, it is a new regression. An old defect that the change made visible needs a recorded decision on who fixes it.
What is an acceptable difference?
A part of the output that may differ between the unchanged and the changed version, agreed and written down before the checks run. The record settles which differences are acceptable in advance and names who allowed each. A difference that changes what a workflow delivers, such as premium rounding, is a decision for the business owner of that workflow.
Which data and environment should the checks use?
The owner of data access decides, and the record names that decision. The checks run on synthetic data or on data this owner has approved, identical for both versions. The environment has to be complete, because missing components can produce false positives or hide issues. AWS also keeps the compared data table free of other applications during the test.
Does acceptance mean the change goes to production?
No. Acceptance decides whether the result is accepted against the record, on the grounds of regression evidence and known differences. Release is a separate decision about when and how the change goes to production, with a release and rollback plan designed for the particular system. An accepted change waits for its own release decision.
Does accepting one change commit us to upgrading the next application?
No. Acceptance covers one change to one application, against the workflows named in its record. Upgrading another application is a separate decision with its own record and checks. In DFG's method, release, data migration, further applications and support are separate purchases, with no obligation to continue after the first accepted result.