UAT Objectives
The key purpose of UAT is not to see that a program or system works according to the specification but to check that it will work in the context of a business or organisation. Many UAT testers are not aware of this and spend their time running tests which should have been properly done in the functional testing part of the System Testing.
UAT is testing the integration of a computer system into a much larger system called the business or organisation. It is a form of Interface Testing and is concerned with checking communication between the system and the users. This does not mean it is a form of Usability testing, which checks how easy it is to work with a computer system. Instead it is about whether a business or organisation can input the information they need to and get back usable results which will enable the business to go forward.
Objectives in Priority Order
UAT Objectives
There are three objectives for UAT that are prioritised in this order:
- Does functionality work in business scenarios?
- Has all functionality been specified?
- Does specified functionality work?
The diagram shows that the first objective is the interaction of customers with staff in a business and them interacting with the system. The second objective shows missing functionality in a dashed box. The third shows the system itself.
1) Does functionality work in business scenarios?
This is the most important objective and is the one on which a system can succeed or fail. The testing involves developing a set of business scenarios which the system is expected to deal with and then running them against it.
Using a Library System as an example one of the scenarios might to "Lend a Book". Developing this Business Scenario shows the End Goal is that one or more books is lent to a borrower. Outlining the main stages for the main goal could be:
- Check the library user's details and books they have out.
- Check the book details to see if it could be lent.
- Create a borrowing record.
This can then be developed into a full Use Case for the scenario. But even this outline indicates a number of paths depending on the number of books a user wants out compared with what they have already, along with failure paths for the various checks, all of which have operational significance.
2) Has all functionality been specified?
Closely tied to the first objective is the question of whether all the functionality has been specified. It is a fact that many of the most important faults in systems is missing or badly specified functionality. And these faults are in the system before coding or configuration of a package occurs. Obviously testing will find these faults, but as they are in the requirements and specifications, they can be discovered earlier in the project by analysing requirements, developing scenarios and reviewing documents.
This emphasises the need for the UAT team to be involved from the beginning of a project and not brought in at the point when System Testing has completed.
3) Does specified functionality work?
Although the main aim is to see if the system works in business scenarios there is still a need to check if the system works correctly. How much effort is used for this depends on the quality and results from System Testing.
If the system is delivered to UAT on time with a complete Test Summary report showing what tests have been performed, the number of Incidents recorded and the outstanding status of those Incidents then you could just check the results of running the business scenarios.
However if the code is delivered late into UAT and there is no Test Summary report then this calls into question the quality of the System Testing. In this case you may wish to conduct extensive testing to see if the system works.
UAT Objectives and the Test Plan
This set of prioritised objectives should be included in the Test Plan as they will show to the system stakeholders exactly what you will do.
In addition it would be wise to cover in the Test Plan the issues of the system being delivered late into testing or being badly tested. This can be done by setting Testing Entry Criteria which ask for the Test Summary report and to have risks and contingencies to cover the issues.
UAT Acceptance
After running all the significant business scenarios a judgement can be made about the system:
- It may have only a few minor faults but be rejected because it does not work well with the business.
- Alternatively it may have many flaws, some requiring workarounds, but is accepted because the business could achieve value from it by using it now rather than waiting for the perfect system.
Using these objectives in this priority order guides the UAT towards producing a business oriented decision about a system.
Other UAT Definitions
- Five reasons not to do UAT and how they affect the project plan.
- UAT Definition is the formal UAT definition.