Workflow Upgrade Testing

This paper makes one argument: you cannot test a workflow end-to-end, and you do not need to. There is a better way, and it covers far more of what matters. None of it will convince you unless you understand the architecture you are dealing with: the engine, the configuration, the gates, and how they fit together. If the workflow is a black box to you, stop here and fix that first; everything below depends on it. Actually doing the testing is a separate paper. This one is the argument.

1. Why you cannot test a workflow with end-to-end cases

People do complicated things in workflows. Transition code piles up on the actions, external configuration gets pulled into the logic, and local conventions grow up around how the data is handled. If you treat the workflow as one big, mysterious block and try to test it all, the problem becomes almost impossible at once.

Look at the maths. A workflow might have ten or so states and maybe twenty to twenty-five actions. Simple. It is a directed graph, so the complexity, per McCabe, is ten to fifteen routes. But the client puts code on a transition, and that code has two routes. That turns one action into two. This is not unusual; these code blocks can introduce enormous complexity. Do the maths, and you can run into 4,000 routes on that same 10-state workflow.

How many test cases can you write for one product? Forty, maybe fifty. Here is the $64,000 question: why those particular cases? What you are doing, perhaps without knowing it, is using your estimate of risk to throw away ninety-nine percent of the possibilities because they are unlikely. You are using a risk estimate to make the problem manageable. How safe is that estimate? It is not. Not even close.

And it gets worse. You have a fixed amount of time and a fixed amount of work. What is elastic? Quality. Quality is the only elastic component. So the bar you have to clear to improve on this is very low. There is another way. Stop treating the workflow as a monolith and start treating it as what it actually is: simple structures sitting in a very solid environment.

2. The fixed points

Two pieces do not move in this story. The engine that manages the transitions does not break. It has run for hundreds of clients, across many instances, for years. The vendor tests it daily. It does not fail.

The workflow configuration in this upgrade is carried across unchanged. The graph of states, transitions, transition code, gates, and their wiring is the same before and after. In case a hand goes up at the back, the workflow configuration screen is just that part of the system. It is not complicated, and it is tested all the time. We are not asking anyone to trust a design assumption. We are pointing out how the system is already used in live environments. The engine is a fixed point. The configuration for this upgrade is fixed.

3. The only parts that can move

Once you fix the engine and the configuration, the workflow can only change behavior if the parts that actually make decisions stop behaving the same way: a rule writes a different value for the same input; a filter returns a different yes/no answer for the same input; or something feeding their inputs has changed defaults, data representations, or the external configuration the transition code depends on.

That leads to one truth. If every relevant rule and every relevant filter behaves identically for the same data, the workflow behaves identically. There is nowhere else for behavior to move.

4. What we are really proving

An upgrade test here is not a fresh correctness review of the business logic. It is a check for behavioral equivalence. The question: given the same object arriving in the same state, does the upgraded workflow manage the same transitions and produce the same resulting data? The claim: with an engine that does not break and a configuration that has not changed, behavior can only differ if the transition code, the gates, or their inputs differ. The strategy: test the transition code and the gates directly, show that they behave the same, show their inputs have not shifted underneath them, and state plainly what lies inside and outside the claim.

The coverage question, and Excel

So let us revisit coverage. I have the workflow configuration, the database, the execution code, the transition code, and the gates — rules and filters. I have shown that the engine, the database, the configuration, and each transition work identically. Do I use an end-to-end approach with one percent coverage? Or do I test all the pieces and establish that nothing is broken at all, which sounds a lot like tending towards a hundred percent coverage? Why would you not do the second? User belief and understanding. That is the reason. So here is an analogy. Excel.

I put a long formula in A1, a few nested IFs, and some other functions. Do you ever stop to test whether RAND() works in Excel? Whether Insert Column works? Do you ever think any part of the product is broken? If the formula is very long, I cannot write it in one cell straight away. I put it across several cells, get each one working, and then assemble them into one. That is exactly the transition code. Testing each cell on its own is easy. Testing them all mashed together is hard.

When Microsoft upgrades Excel, do you test the sheets for the actual application functions? No. Why not? Because 90% of broken spreadsheets are due to user error, not Excel. It is the same scenario here. Your risk estimate says Excel is fine to about 99.999 percent. The estimate I offer is not 99.999 percent; I would make it, but no one would believe me. What I am offering is a testing method that gives you well over ninety-nine times the coverage of end-to-end testing. Here is the $128,000 question. Show me that I am wrong, or maybe less. Show me how you can beat the coverage using end-to-end testing.

5. Core artifacts that we deliver

The approach produces a small set of artifacts sitting behind a one-page justification, enough to keep the business story short and the technical story solid. A workflow component inventory: states, transitions, gates, transition code, and any known external dependencies. A read/write map: for each transition, which fields its gates read and which fields its transition code read and write. Dependency and coupling notes: where a field written upstream is read downstream — an analysis, not a separate testing method, to show we have not missed part of a chain. A differential test set: the transition code, gates, defaults, data representations, and external configuration that the version change might affect, derived by cross-referencing the release-note changes against the configuration. Rule and filter test packs: evidence that each relevant rule and filter behaves the same across versions. And a boundary statement: given the same object arriving in the same state, the workflow does the same thing to it; what happens outside the workflow boundary is out of scope.

We care that each transition works before and after the upgrade. We do not have to prove that a particular trade will start at state A and arrive at state F. If the workflows are identical, it will, as long as the external data changes are consistent too. That is the statement. We decouple the changes in trade data from the workflow itself. On a risk basis: workflow code rarely breaks, think about it. Filter structures really do not break. If that sounds too strong, just ask yourself whether you have ever seen one break. It is extremely rare.

6. How we test transition code and gates

Transition code and gates are the only parts that can change the behavior, so that is where the effort should go. Transition code: for each rule in the differential test set, build inputs that cover the normal cases and the edge cases. Use the rule documentation and the audit trails to confirm that, for the same inputs, the rule writes the same outputs in both versions. Gates: for each filter, identify the fields it reads. Group inputs into “yes” and “no” sets, pick examples, and test the boundary between them — off-by-one comparisons, empty values, dates, and defaults. Confirm that, for the same inputs, the filter returns the same yes/no result in both versions. Inputs and representation: where defaults, type conversions, or representations have changed, test that the filter still effectively sees the same input.

The read/write map is the safety net. It makes sure every rule whose output feeds a filter, and every filter that depends on those outputs, is included. It does not add a new kind of test; it stops a dependency from being missed. Actually testing them is a different paper. This one makes a single argument: you cannot test a workflow end-to-end.

7. Why we do not try to test every route

Testing every possible path through a non-trivial workflow is not realistic. I am making the point again, because getting over the belief that workflows are inherently complicated is the hard part. The number of combinations explodes as the transition code accumulates, especially when execution order matters or the code relies on external values. Component-level proof only works on the parts that can change behavior. It does not claim the workflow is “correct” in some abstract sense. It shows that, across an upgrade, the workflow behaves the same way it did before. I will keep saying that.

8. What we deliberately do not test

I have to state what is out of scope. We do not retest that a filter manages the transition it is attached to — the engine handles transition management, and it does not break. We do not test multiple values inside a group that a filter cannot tell apart; one well-chosen example covers the group. This whole system is behavioral, not functional. People say it is object-oriented, and then forget it the moment they start testing. We do not test field combinations that no single filter and no dependency chain ever examines together — nothing in the workflow can see them jointly. And we do not use trade replay as the primary method; we might look at it in one or two cases, once the transition code and gates are proven equivalent.

9. Practical order of work

Build the workflow component inventory and the read/write map, and identify the dependency chains. It sounds like a lot, but if you can query the database or run a report, it should be straightforward. Cross the release-note changes against the configuration to derive the differential test set. Test each rule and each filter in that set for equal behavior across versions, using the new version’s actual data — a behavioral exercise, not a functional one. Pay extra attention to automatic dependency chains and external configuration dependencies. And here is a free one, a never-do-it check: any case where more than one transition can be true at once, or where there is no other case. Something could get stuck. That is just bad practice. Fix it. Write the boundary statement: same object, same starting state, same result inside the workflow. Keep trade replay as a limited, secondary check, not the core proof.

10. One-page justification (for business readers)

The workflow engine does not break, and the configuration for this upgrade is unchanged. The only parts that can change behavior are the transition code, the gates, and the data that feeds them. If every relevant rule and filter behaves the same for the same data, the workflow behaves the same. We build a read/write map so no dependency is missed, and a differential test set to put the effort where the upgrade can actually touch behavior. We test transition code and gates directly, including the boundary and representation cases, and we state plainly what is inside and outside the claim. We avoid “test every route” strategies, because they do not scale and do not add meaningful assurance.

That is the whole of it. Believing the argument needs the architecture; doing the testing is the next paper.