One of the final stages of development tends to be UAT – User Acceptance Testing, where end users are presented with the final product and asked to test and sign off on the product.
In a recent session it struck me how lubricious it was to combine acceptance testing with system testing. Expecting an end user to instantly identify not just UX problems, but formula problems in a very fast half hour meeting. Without being intimately familiar with the underlying data how could they spot all issues – spotting 5.0% when it should be 7.4%.
System testing is taken care of by Unit Tests and ATDD in the hopes that any issues with the underlying logic will be exposed and corrected. These should be based on the requirements that the team received, and these should have been signed off by the business sponsor.
Course what we tend to find is that users submit requests that are more similar to wish lists than well defined requirements. Now as developers work on these requirements, we start creating unit tests based on our interpretation of the requirements. If there is disagreement on the team about the interpretation, then we seek the business user to confirm.
Yet if there is no disagreement on the interpretation then the formulas get implemented and the first time the user gets a chance to spot the issue is during UAT. Now we have a bug in the system that got full sign off from everyone and will remain in the system until someone notices the difference.
So now the question is how could we have seen this coming and better prepared to prevent last minute changes or introducing bugs…