top of page
Search

A New Test Run Hub Is Not a Test Strategy

2 days ago
2 min read

Recently I was asked whether a better Test Run Hub in Azure DevOps would improve UAT.

I think it will. I do however think people can expect rather too much from the tooling.

A dashboard can tell me a test passed. What I really want to know is what we proved by passing it.


Take something as simple as employee dates.

A test might pass because the date appeared in the new system. Nobody checked whether the format was UK rather than US, so 06/10 is suddenly June rather than October.

A calamity waiting to befoul us.



The test passed. The outcome is wrong.

That is my gripe with headline pass rates.

196 tests passing out of 200 means very little until I understand the other 4. If those failures concern payroll calculations, integrations or permissions, I am potentially more interested in the 4 than the 196.


For me, good UAT comes down to 3 questions:

  1. What were we trying to prove? 

  2. What evidence proves it? 

  3. Who accepts the remaining risk? 


Azure DevOps can organise the test execution, results, evidence and defects. A controlled HR/payroll UAT model then needs to carry that evidence through to the go-live decision.


Quick UAT check: Take your 3 highest-risk processes and trace each from requirement to test, evidence and decision owner. If you cannot complete that chain, I would fix that before worrying about the overall pass rate.

 I like anything that makes evidence easier to find and failed tests easier to follow.

I do however see the Test Run Hub as supporting the testing process. Good requirements, meaningful tests and clear ownership still determine whether that evidence is useful.

A better view of weak testing is still weak testing.


 
 
 

Recent Posts

See All

Comments


bottom of page