Software testing is not about hunting bugs but delivering great software. However, a vast majority of project managers still consider bug count as the most important measure of a tester’s effectiveness. In my career, I’ve come across a host of metrics which can be used to measure the effectiveness of a tester in delivering the perfect software.
A well designed tester’s Key Performance Indicators should show performance in terms of goals for different functional areas, overall performance as a team, and customer’s feedbacks. Here I am presenting a few of the KPIs we follow in our team, which resulted in an improvement in customer satisfaction index.Requirement knowledge : This metric indicates how effectively a tester understands the requirement. If the tester is not 100% aware of the customer’s requirements, a lot of time could be wasted in posting invalid bugs. This can be checked by evaluating the test cases or test plans prepared by a tester with reference to the requirement document. In our team, we have set a target of 100% conformance to Software Requirement Specification document for the test cases.Test coverage : This metric indicates how many of the requirements are converted into test cases. This metric ensures that there will be continual improvement in the thoroughness of testing with respect to the requirements of the customer. This is measured as the percentage of test cases covered or executed by a tester. This should be analysed from the results provided by testers in the test case document. In our team, we have kept a target of 95% or above for this metric.Testing Rework : This metric indicates how many bugs were reassigned back to the tester due to lack of information. This includes the cases where developers ask for more information regarding a bug. This happens when a tester is unable to provide proper steps to recreate the issue or give proper screenshots, and leads to wasted time where a re-test needs to be done. In our team the target is set as 0 testing rework cases. Customer complaints :This metric will indicate the satisfaction index of the customer. Each bug reported by the customer indicates a test case the tester missed to execute. So keeping a watch on the number of complaints, and resolving the root cause of the complaints will lead to a process which produces great software. In our team the target is set as 0 complaints.Bug Success Rates : This metric indicates how many of your bugs resulted in a valid fix. Invalid bugs happen when SRS is not correctly interpreted, or the testing is wrong. Invalid bugs put a strain on the dead lines for a project, and needs to be avoided. In our team, the target is to achieve 0 Invalid bugs. and finally, Team performance : This is a collection of metrices that should set for the team as a whole. Eg: milestone delays, start lags, response times, etc. Including the total team metrices in tester KPI is important as the testing team needs to work together with others to make sure the overall targets are achieved. By including the team targets in the tester’s KPI, the testers will do whatever is needed from their part to get the project finished on time. It will encourage the developers to provide testability in their code, and the testers to communicate with developers about possible vulnerabilitiesDon’t ignore the fact that every metric can be manipulated. So, track maximum data I from a bug tracking software (I suggest JIRA ) and through effective communication. Keep them in perspective.So, in conclusion, there are a variety of metrices by which you can measure a tester’s performance. Monitoring and continually improving these metrices will help you deliver great software, and result in good brand image.
Please post your valuable comments below.
About the Author :
Sharidas. H joined Bobcares in May 2011. She has been working as a dedicated software tester for more than a year and is very interested in researching new testing methodologies. In her free time she listens to melodies.
So, the more testers you have; the better your coverage ( since you can get more testing done), the better your understanding of the requirements ( since you can as a team, understand them in greater detail) … correct? So, will increasing the number of testers increase software quality?
Hi Deepu,
Thanks for posting your comment.
These points are all about increasing software quality by improving each tester’s capability through periodic KPI analysis.
To decide developer to tester ratio, you need to consider other factors like methodology followed, experience of developers etc.
Once you fix this and recruit them, you can measure and improve individual performances of each tester through these KPIs. These lead to an improved team performance and the KPI ‘Team performance ‘ aims to ensure complete involvement of each tester in achieving this.
Thanks!
Nice work…This article gave a better ideas. Keep on writing….