December 3, 2017
5 User Acceptance Testing Best Practices
UAT is Not a Surprise Party. At least not one I want to be invited to.
I was sitting in a recent IRT conference when the topic of User Acceptance Testing (UAT) was addressed. The speaker said they use UAT as a process to find bugs in the software, to ‘validate’ the system. I looked around the room and saw heads nodding. Huh? How did we get here? When did UAT become a surprise party where the trial sponsors used this as an exercise to find bugs and quality issues in the software?
The vendor is responsible for ensuring the system is properly validated to perform to requirements. The sponsor needs to accept the system for use and this process, UAT, involves having the sponsor interact with the system and signing-off that it works as intended – or is fit-for-purpose.
So, does fit-for-purpose mean the software works? Or does it really mean that it performs to your study specifications laid out in your protocol? If we can all agree on the latter, let’s stop viewing UAT as a bug finding expedition and start raising the bar on the system that is presented to you at UAT. Do you, as a sponsor, need to demonstrate oversight of your RTSM (randomization and trial supply management) vendor and their processes? Absolutely. Do you need to ensure that critical systems like RTSM are correctly supporting your studies? Of course. Do you need to revalidate the RTSM after your vendor does to prove this? No.
Easier said than done? Sure, but here are best practices for getting the most of out of your UAT process – and focus your efforts where they should be.
1) Align Expectations on the UAT Process
Given the above, the first step is to understand your organization’s definition and process for UAT. Who is involved? Who drives the process? What level of UAT is currently done – it is too little? Too much? It is also a good time to sit down with your RTSM project team and look at historical data. What issues (or surprises!) have come out of UAT in the past? Could some of them been avoided? Now is the time to define the process and manage expectations.
2) Understand What Level of UAT is Necessary
The process for UAT testing varies widely across the industry. Some organizations don’t conduct UAT at all while others have teams of people writing scripts and essentially revalidating the system after the vendor does. There needs to be a balance. With a 100% configurable and fully cloud based RTSM, UAT can be directed at the critical functions of the system as well as any new functionality. Major findings should be a red flag to overall quality issues. The process of UAT should be used to bring to the forefront additional needs that the sponsor may not have thought of prior to seeing the system, not serve as a software testing exercise .
"'How much validation is enough?' is a question that all sponsors struggle with. By focusing on those areas that introduce the highest risk to the study; particularly those that are not sufficiently covered by the vendor - should help sponsors hone in to exactly what they need to do to complete the validation of their RTSM."-Laura Araujo, Vice President, Quality, 4G Clinical
3) RTSM SME Should Drive the User Acceptance Testing Process
To ensure your user acceptance testing is executed as efficiently as possible, this process should be managed by your RTSM project lead or SME. As part of this role, the RTSM SME should pre-schedule testing with all participants. This way, the participants have a dedicated period of time to spend in the system testing and won’t bypass or rush through this important step in system go-live. A well thought-out and executed UAT lends itself to a higher quality product. Don’t have an RTSM SME in your organization? That is absolutely okay. In that case, identify a single person on your study team that is responsible to ensure UAT takes place and look to your RTSM vendor to guide your teams regarding UAT best practices and process.
4) Actual RTSM Users Should Participate in UAT
- You don’t need everyone that will ever use the RTSM to participate in UAT. However, at minimum there should be representation from all user groups including:
- The RTSM project team that helped define the requirements of the system.They can determine if the requirements they documented are represented accurately in the system. This team often includes clinical operations and clinical supply leads for the study.
- Actual system users, including the study team, sites and CROs.
- This is the one that is skipped during UAT, but is critical. The requirements as laid out by the RTSM project team can be reflected in the system, but if it is not user-friendly you don't want to wait until system go-live to find that out.
5) UAT Should be a Final Check Not a Bug Finding Exercise
As technology has advanced, sponsors now have the ability to see a fully deployable system before UAT. With the use of agile software methodologies, customer feedback can be incorporated into the RTSM development process. Instead of receiving the system right before UAT, the sponsor can see the system before signing the specifications, and in some cases, in a demo before seeing the actual spec for the first time. The benefit to the customer is the ability to interact with the system and provide feedback during several iterative process loops. They know what system they are getting well before they sign-off on the specifications. The increased transparency and customer involvement from receipt of the draft protocol ensures UAT will be a final check and there will not be delays to their study.
"The RTSM vendor begins the validation process with the goal of reducing the sponsors overall risk of using the system. Proper validation and testing by the vendor should ensure the sponsor validation is simply a final check that everything is in place. UAT shouldn't be a surprise party if the vendor does their job upfront with testing and validation."
-Laura Araujo, Vice President, Quality, 4G Clinical
Bottom Line: The Conversation around UAT Needs to Change.
The above best practices are important but to be truly effective the conversation around UAT in the industry needs to change. If sponsors continue to accept UAT as a method to find bugs and quality issues, the process for delivering systems will not evolve. If we truly want to change this thinking, RTSM vendors must also do their part and provide systems for UAT that do not have quality issues. If there is one thing I can leave you with is to change your expectations around UAT and RTSM system delivery. The technology is there and quality should not be in question as you approve your system as fit-for-purpose.
Looking for more best practices? Watch our webinar on Best Practices for User-Acceptance Testing (UAT) in Clinical IRT/ RTSM Systems
Tag(s):
RTSM