The Better Solution for UAT – Real World Insight to Ditching Those Excel Files

“There has to be a better way to implement UAT at our company.”  If this sentence resonates with you, checking in with testing managers with many years of experience who have faced the same challenges as you face now could give you the insight you seek.  Let us turn to Wagner Canto of ABB Brazil.  A few years ago, when he was using Excel files to manage his User Acceptance Testing, he sought out a better solution that could save time and money.  Here is Wagner’s story.

Wagner Canto, manages Information Technology Projects and Services for ABB Brazil, a company of 3000 employees in the automotive industry.  They have 3 manufacturing plants, 4 business divisions plus a 5th division that supports the 4 divisions with administrative and IT services.  The IT department services 2000 SAP users and 50 SAP key-users, with a dedicated SAP IT team of 35 members.

Where did your UAT journey begin to get there?

Canto: ABB Brazil’s SAP system (version 6.04) was implemented in August 2012.
After the implementation, we executed a stabilization project in order to implement several processes improvements. The stabilization project was completed in August 2014.

We used Panaya for the first time in April 2014. It was introduced by our colleagues from ABB US that were already experienced users.
At that time, Panaya was very different from today’s version. It was only a record & play tool, useful, but not enough to manage a big testing project.
Then Panaya team started to develop a new version of the tool and in November 2014 released the “Guided test” version which replaced the old version.
In December 2014 we executed a big test using the new Panaya version and the result was very good.

This block renders a quote for the post drawn from the post's custom fields. Modify the quote below the content editor in the Quote fields.

Since 2014, the whole SAP team and key-users learned how to use Panaya (no formal training was required) and today Panaya is used to perform all SAP tests in ABB Brazil.

What is UAT testing?

Difficult to give up what works, even if it is time-consuming process

Canto was originally using Excel and Word documents as a basis for his UAT, as many enterprises continue to do until today.  He worked from a large Excel file, sending out smaller Excel files to his testers, collecting the information and then updating his main Excel file.  This, he realized, took an extraordinary amount of time, much of it being wasted on chasing his business user testers scattered in different locations.

• To improve the situation, his specific objectives were to:
• Spend less working hours (specially from the business people) to complete a user acceptance test (UAT);
• Find the maximum number of errors during the test and fix them as soon as possible;
• Avoid serious post go-live incidents that causes interruptions in the business processes;
• Create an organized database to store:
  1. The testing scenarios to be re-used.
  2. The test results (evidences) to be audited.
• Implement a testing process that can be replicated every time that he needed to perform a test. This testing process needed to be user friendly enough to be easily understood by non-technical people, but on the other hand, also needed to follow the software testing best practices;
• Implement a set of automatic regression tests that could be executed every time he had a sensitive system change. This was a big challenge due to the complexity and the number of business scenarios to be tested. Also, depending on the change, the test needed to be prepared specifically to validate a certain part/process/functionality of the system.

Have your views on testing changed over time?

Canto:  A lot. Being honest, I was a bit skeptical about testing tools in general. Before Panaya, I had some experience of using SAP eCATT (native SAP testing tool) without much success. Many tools are focused only on the record & play feature. Usually testing tools don’t have any support for organizing the test.
After some time using Panaya we realized that the purpose of our testing process is not just to find system defects, but also to validate and confirm the requirements and to train the users!

In terms of using Panaya for testing, what do you find to be the best feature for you and how has this helped your team?

Canto:  For sure was the “Guided test” feature released in December 2014. This feature “retired” the spreadsheets and Word documents used to control the test execution and collect test evidences. It also provided an excellent support for ERP test management, including planning, execution and follow-up. Due that we could execute the test with less technical resources. I think that we can’t describe all benefits that we achieved just in numbers. There are also intangible benefits that are difficult to measure such as the quality of testing documentation, clear and intelligible testing process, testing script database (to be re-used), reduction of post go-live incidents and less time spent by the business areas and test management/control.

The functionality to assess the impact of EHP upgrading and plan the tests of a new SAP version is also very useful and can save a lot of time (and money).

From Excel to end-to-end test management and execution = Simplification and Acceleration

Relying on tools that are unmanageable and unscalable is risky and time consuming for test managers, technical testers and business users. Organizations need a solution that embraces the collaborative part of execution, the tedious part of documentation, and the increasing need for real-time visibility, to increase efficiency, quality and certainty of testing.

Panaya Test Dynamix delivers a new way of managing and executing functional test cycles so that organizations can validate their changes are delivered without risk. With Panaya, test managers have the confidence for go-live by not only testing right, but also testing the right things. Both Technical testers and key business users have an easy to use solution that greatly increases adoption with collaboration and documentation to get it done seamlessly. Testing is simplified and accelerated.