Insights into how technology and the outsourcing of R&D are changing the software industry

I always liked this military term.  For those who have been living in a bunker for the past 10 years, Force Multiplier refers to a factor that dramatically increases the combat-effectiveness of a given military force.

In the world of software testing, a strong and well executed test automation program can act as a force multiplier for not only your QA organization, but the entire R&D team.  With an aggressive test automation program you can:

• Reduce test cycles by at least 40%
• Slash testing budgets by removing the time and effort from manual testing
• Vastly expand code coverage

But where automation becomes a real force mutlipler is in catching more bugs before the code is released into production!  Industry data shows that the cost of remediation can be 100x higher post-release than at the beginning of the product development cycle.  By using automation to find the bugs earlier in the cycle you can greatly reduce re-work (not a value-added activity in the first place) and sustainting engineering costs.  On top of that you can cut support volumes and costs by up to 75% (Gartner), not to mention saving yourself the damage to your brand and CSAT by having bad product in the field.

Even just on a pure QA testing budget perspective, Symphony Services is driving ROI’s of 2-3 months, saving in one case about $100K/month in testing costs with our test automation services.

So it all sounds great, right?  Yet when we engage with clients and prospects we find that while some have some level of automation, it’s often quite low, with less than 25% of all test cases automated.  And of course many others haven’t implemented automation at all.  Why is this?  I’ve got a couple of ideas:

1. Lack of Automation Expertise: This is of course the most obvious reason why people don’t use it aggressively.  But that shouldn’t mean that you should eschew test automation as a strategy.  There are plenty of companies like Symphony Services who have the right expertise, and automation frameworks to help — although not as talented
2. Wrong Automation Strategy: This of course pre-supposed that there’s a strategy in the first place and not a decision to just start automating cases.  This problem manifests itself through Wrong selection of test cases, inadequate maintenance considerations, insufficient ROI models lead to inefficient test automation with low ROI yield.
3. High Upfront Automation Costs: That is if you’re starting from scratch.  Highly maintainable scripts require careful planning, development of reusable framework & components.  If you don’t have these handy, this increases upfront investments and reduces time to value.  Additionally, you need to drive towards lights-out execution to really ratchet up the benefits, but the ability to do that needs to be designed in at the beginning.
4. No Attention to Script Maintenance: This is what usually kills an automation project.  Teams begin an automation project, but then can’t keep up with the script maintenance.  So the scripts get out of date, the benefits begin to melt away and the project seems like a failure and is abandoned.  Common approaches such as Record & Replay yeild fragile automation scripts and don’t work.  You need to access re-usable libraries and modular, self-sufficient scripts.

So what’s holding you back from implementing test automation?  Of course if I’m all wet, let me know that too.

Please write back and let me know your experiences with test automation.