Software Testing Automation is a practice that makes absolute theoretical sense to all C-level executives because it hits the perfect three-part harmony of higher quality in reduced cycle time for decreased cost, music to any C-level’s ears, regardless of her/his favorite genre! Like any initiative, putting Software Testing Automation theory into practice is easier said than done and the difficult thing to predict is the size and timing of the ROI, which you really need to know BEFORE you take the plunge.
There are a bunch of CAPEX and OPEX items to consider and balance against the age old manual testing effort including.
- Software Automation Tools
- Automation creation effort
- Automation cycle execution
- Automation maintenance
The cost of the software automation tools will vary given the application platform and technologies used to develop the system, however, it is fairly straight forward to calculate the cost of tools, as it is with a developer tool set, once you figure out which one is the best fit to address your specific technology needs. You can also spend a lot of time and effort trying to make this determination yourself, or you can get some help and get to the right answer for you, faster.
The more difficult part in determining the ROI for software testing automation is the resource effort to get it up and running and then maintain it.
GlobalNow has put together a Software Testing Automation ROI Calculator that helps provide perspective on the magnitude of the savings and the timeline. The calculator comes with some default values reflecting GlobalNow experience data that can be changed to reflect your specific situation. The results displayed provides a foundational understanding of the cost improvement opportunity considering only the tangible numbers for both approaches.
Some notes about the Automation ROI Calculator and Exclusions:
Sometimes organizations short cut the testing they do on their applications, so when you use the calculator, I suggest that you run it at least twice. First plug in the numbers that reflect what you are doing today and second, plug in the numbers that best practices indicate you should be doing. This will help you see the goal posts of how much automation can impact your results.
Automation tool costs are excluded in the ROI Calculator. These costs can be anywhere from $0, using open source tools, to a lot, depending on the tool vendor and your application environment. Many organizations use open source tools, so there is no CAPEX requirement, but they don’t always address all the automation requirements. What is clear is that if there is a tool purchase, the cost will be front loaded therefore extending the ROI period. GlobalNow can help you with this assessment and decision. (Note: when working with an outsourcing QA provider (such as GlobalNow), there may be an option to include these costs in the providers fee structure and not as a capex requirement).
Developing test cases and test data must be done regardless of how the test cases are executed. So these costs are considered a wash and not included.
Intangible benefits of faster speed to market and higher quality gained from executing all required tests every time are also not considered in the Automation ROI calculator. With automation, product enhancements can be delivered faster due to the reduced cycle time for test execution. Plus, under manual testing practice, QA time is frequently compressed from the planned time allotment due to development complications. This compression is usually compensated by reducing the test coverage during the test cycle, which nearly guarantees production defects. The impact of a production defect can be significant. Identifying and resolving a defect ripples through the organization in the form of operations investigation, ticket management, client escalation and finally the development effort to fix the root cause, which also impacts the time available for new, revenue generating features and projects. Putting a number on this is nearly impossible, but we all know it’s a significant cost to the organization, and would likely contribute to reducing the ROI period.
The calculator focuses on the main levers of software testing, including number of test cases, browser support, mobile support, resource cost and the number of product releases over the course of a year. Some other key levers are associated with the process itself, specifically the effective time to execute a test and automate a test as well as the rate of change in the application from release to release.
Try out the Software Automation ROI Calculator using some of your own metrics.
SIgn up