Measuring the success of TCoE

“What gets measured, gets done”. This could not be truer after setting up a Test Center of Excellence (TCoE). A TCoE is setup to centralize testing resources, tools and practices in a large enterprise and offer testing as a shared service across multiple groups. Enterprises need to constantly prove the value of retaining the TCoE and increase the management’s commitment to further develop the TCoE. In this paper, we out-line the different metrics that measure the business benefits delivered by the TCoE.

Why TCoE?

Traditional “siloed test teams” are distributed across multiple business units (or) groups and they are governed by different managers. Because of this organizational design, test teams cannot ensure testing consistency across the organization as they are controlled by managers who have their own priorities and political realities about the importance and budget allocation for testing. This led to massive inefficiencies across the organization in terms of costs, reusing best practices, quality of testing, effective utilization of teams, hardware & software. A testing centre of excellence solves this problem by the centralization of all test assets (people, hardware infrastructure, software licenses, process assets) across the organization and offering “testing” as a shared service across multiple lines of business units (or) groups. Collapsing multiple test groups into a centralized testing group will lead to better efficiencies due to economies of scale & optimization. Testing Centres of Excellence (TCoE) (Ref Fig 1.2) framework ties all the strategic assets of the testing such as people, process, hardware & tools which are held together through enablers and a robust governance model. Transformation of a “siloed testing organization” to a “TCoE based testing organization”.

Measuring success of TCoE

There is a need for a measurement framework that can compare the intended benefits versus actual benefits to demonstrate the effectiveness of TCoE. It should be comprehensive and cover all aspects like people, costs, quality, test maturity and speed. A TCoE promises the following benefits for an organization.

• Reduction in testing costs
• Improved quality of applications in production
• Improved test maturity
• Better productivity
• Improved time to market
• Improvement in overall skill-set of the team

Based on these six benefits and our experience of working with enterprises, we recommend the following framework to measure the effectiveness of a TCoE (Ref Table 1.1)

• Testing Cost
  1. Average cost of testing per hour
  2. Resource utilization
• Quality of Planning
  1. Planning performance
• Quality of Product
  1. Defect leakage into production
• Test Maturity
  1. Test maturity rating
  2. Test process standardization index
• Competency of the team
  1. Skill-set index rating
• Infrastructure Cost (hardware & software licenses)
  1. Over all infrastructure costs
  2. Utilization

Conclusion

Many enterprises have established testing centres of excellence to improve the release quality of the applications (or) products in production environment, deliver better business value to the internal customers of testing, improve the overall maturity of testing & optimize the testing costs across the enterprise. Unless the promised benefits are measured against the actual delivered benefits, the CIO/Head of Testing will not be able to demonstrate the effectiveness of the TCoE. This framework can help the IT leadership in an enterprise to measure, showcase and in some cases figure out what needs to be done to really deliver the intended benefits of a testing center of excellence.