Which SAP testing methodology are you using? Are you sure that it’s the right methodology for your organization? And, even if it is, could you improve it?
We’ve researched thousands of SAP ERP landscapes and found that most organizations tend to use one of three different methodologies. We’ve labelled them Ad-Hoc, Organized, and Scripted.
As you can see, most IT departments fall into the ‘Organized’ category.
So which approach do you take…and should you reconsider?
What is the ‘Ad-Hoc’ Testing Methodology?
The Ad-Hoc testing methodology is generally adopted by smaller organizations. In these cases, you’re relying on business process experts and working very closely with them on a daily basis. Tests are run as and when needed, there is no test catalog, and test executions don’t need to be documented.
But there are challenges with this approach – you’re dependent on the knowledge of specific employees, which exposes you to a certain risk level. For this reason, you may spend time documenting your business processes in detail, which is tedious, solely to ensure knowledge persistency in the company, rather than to support the test process itself.
Should you move to ‘Organized’ or ‘Scripted’?
The ‘Ad-Hoc’ approach is fine if you stay small enough to contain the testing requirements, while ensuring the same quality of delivery. But you will need to reconsider this if your company scales up or needs to comply with regulations like SOX or FDA. When this becomes the case – or if you just want to improve your process – you might want to invest in a test management tool to reduce the overhead of the testers and overcome the change management challenges.
What is the Organized Testing Methodology?
If you’re taking the ‘Organized’ approach, you’re also relying on the knowledge of the business process owners to execute the test. However, you’re managing and tracking the testing process, handle complex test cycles involving end-to-end testing, documenting test scenarios up to the transaction level (high level instructions), and probably aren’t using test automation very much.
Not surprisingly, this methodology requires a lot of work and commitment from your functional resources and your business users. In End to End Testing, they are all strongly dependent on one another so if they fail to document a step – it jeopardizes the entire test process.
Keeping control and visibility over the test progress is also a challenge, especially in decentralized environments. Unfortunately, test management tools like HPQC, Solution Manager or IBM Rational don’t eliminate these challenges entirely.
Should you Move to ‘Scripted’?
Because of this, the temptation to use more automated testing or move over to ‘Scripted’ can be immense. However, I think that you need to consider the real ROI of moving to such a solution. It’s true that you will reduce the time spent on test execution. But this is just an illusion. The time spent on test execution just shifts to knowledge transfer and governance over the third party – whether it’s an automated tool or an outsourced service.
Before, you could rely on your own people to do the job, now these people need to invest time in explaining it to other people.
Before, you relied on your own methodologies and people, now you have to control also the third party.
Therefore, if you’re taking the ‘Organized’ approach, you should keep on using the same people to test – but maybe look for tools that will enable you to increase their productivity. Focus on what they really need to do – and take away the overheads. There’s a lot you can do to improve your processes without taking the leap to ‘Scripted’.
What is the ‘Scripted’ testing methodology?
With the ‘Scripted’ methodology, the person performing the test isn’t the subject matter expert. With this approach, the IT department relies on QA teams, System Integrators, or Offshore resources to conduct their testing, using automation tools or not. The testing process is managed and tracked, test scenarios are documented to the screen/field level (very detailed instructions), and automation is often used.
This approach liberates your subject matter experts from time-consuming testing processes. However, they still need to ensure a smooth and accurate knowledge transfer to the person or the tool performing the test. And you need to guarantee top quality – which isn’t always easy given the limited visibility into testing that this approach offers.
Should you Consider Another Methodology?
There are clearly challenges with this approach. You don’t have much visibility into the testing or change management process. Plus, the time it takes to transfer knowledge from the business users to outsourced resources is often forgotten or overlooked. But it is a real cost.
When certain business processes need to be checked on a daily or weekly basis, you can’t really afford having your business experts spend their time testing them, and therefore being “scripted” would be a must for these processes.
In my opinion, there’s no need to change methodologies here but you should start checking out solutions that will enable you to reduce both the time you spend sharing knowledge and documenting test executions. If this can also be coupled with test acceleration, you can be confident that you’re all working in a fast, efficient and standardized manner.
So are you using the ‘wrong’ testing methodology? Is there such a thing? The truth is that there is no one ideal testing methodology. However, there are approaches that might suit your particular organization more than others. More importantly, there are many tools and processes that you can implement to improve your approach – regardless of the specific methodology you’re using.
Which methodology are you using and how do you overcome the challenges involved?