5 Reasons SAP ERP Test Automation DOES NOT Cover All Your Testing Needs!

Think SAP ERP Test Automation Tools are the answer to all your testing needs? Think again!

It’s true that test automation tools are designed  to dramatically cut costs, manual labor, and shorten testing duration. And certainly, using an automated tool for your SAP testing is a logical and valuable course of action. But it CANNOT be the full picture.

Looking at organizations around the world (and I’ve worked with a lot) there is a clear limit to the level of functional test automation.

Most companies are somewhere between 0% and 30% of automated test coverage, and there are good reasons for this limited level of automation.

But why? Well, I have some ideas.

Here are 5 Reasons SAP ERP Test Automation DOES NOT Cover All Your Testing Needs:

1.  Testers’ Skillset

Most automation tools offer the option to “record” activities on the screen and then “play” the recorded script back. For very simple, single transactions this can be achieved, but to create a real end-to-end scenario this method is not enough.

You MUST have scripting and programming skills to create the proper automated test script, as there are too many adjustments you need to make to the original script.

Let’s take the example of testing a purchase order (ME21N), followed with another transaction that requires the PO number (like MIRO).

As the PO number is changed for every execution, you must create a dedicated script to move the value between the transactions. Variables, case conditions and different execution paths based on values should be scripted.

2. Domain Knowledge

Having a test engineer that knows SAP, knows the company business process and has the ability to write automated test script is not simple. It is also quite expensive.

3. Test Script Maintenance

SAP is a very dynamic application. Transports of changes are done on a weekly/monthly basis, both for new functionality and for maintenance activities.

When the application is changed, the testing script should be modified – after all, it’s likely that quite some time has passed since it was written and the test developer may not even be with the company any more. Keeping thousands of automated scripts up-to-date is an extremely challenging task.

Another brier for proper maintenance is lack of test script documentation.

Throughout the years we have developed good practices to document code we are writing (so developers following us will understand what we wrote), but these practices are not common yet in the automated test script arena.

Test script is preserved as an ad-hoc activity and the resulting lack of documentation makes maintenance even harder.

4. Changes in the Team

Many organizations use system integrators to build their automated test script library, leveraging the expertise of the SI test engineers. As the team that manages the scripts is not the same one that wrote them, there are gaps in business knowhow and familiarity of the test code.

5. Test Master Data

To run tests, you need to have the proper test data in the tested environment.

For example, to use any material you must have it in stock – or SAP will stop the flow of the transaction.

When you run the automated test in many cycles, the master data must be constantly handled, and this is a very complex task (the data in SAP is spread over many entities, with different connections between them).

Test automation is an excellent concept, and there are places where it is very useful for dramatically shortening the time to market. But because of the limitations mentioned above (and many others) it’s only part of the solution.

The majority of testing for SAP must involve manual tests and, as such, mature organizations combine both methods.

Whichever approach you decide to adopt, one thing is perfectly clear: aiming to use automated tools as the sole answer to all your testing needs just doesn’t work!