| Table of Contents | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| minLevel | 1 | |||||||||
| Table of Contents | ||||||||||
|
KPI Name / Definition | Representation | Tool | Maturity Levels | KPI Calculation | On Hover | Remarks | Status||||||||||||||||||||
Defect Injection Ratemeasures the total number of defects (bugs) linked to closed stories which are part of a Iteration. | Line Graph
| Agile Project Management
| M1: > 175% M2: 175% -125% M3: 125%-75% M4: 75-25% M5: 25% | No. of defects tagged to all stories closed in an iteration/ Total no. of stories closed in the iteration.
| Sprint Name: <<Percentage Value>> Defects: <<Value>> Closed Stories: <<Value>> |
| Definition
Rep to On Hover -
Remarks -
|
| ||||||||||||||||||
First time pass ratemeasures the percentage of tickets that pass QA first time (without stimulating a return transition or not having a linked defect) | Line Graph with multi-select filter
| Agile Project Management
| M1 <25% M2 >=25-50% , M3 - >=50-75% , M4 - >=75-90% , M5 - >=90% , | No. of issues closed in a iteration which do not have a return transition or any defects tagged/ Total no. of issues closed in the iteration.
| Sprint Name: <<Percentage Value>> FTP Stories: <<Value>> Closed stories: <<Value>> |
| Definition -To check Rep to On Hover -
Remarks -
|
| ||||||||||||||||||
Defect Densitymeasures ratio of the total number of defects (bugs) linked to closed stories and the size of a closed stories | Line Graph
| Agile Project Management
| M1: > 90% M2: 60% -90% M3: 25%-60% M4: 10-25% M5: <10% | No. of defects tagged to all stories closed in a sprint/ Total size of stories closed in the iteration
| Sprint Name: <<Percentage Value>> Defects: <<Value>> Total Size of Closed stories : <<Value>> |
Definition
Rep to On Hover -
Status | | |||||||||||||||||||
| colour | Green | title | DONE||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Defect Seepage Ratemeasures the percentage of defects leaked from the current testing stage to the subsequent stage | Line Graph
| Agile Project Management
| M1 - >=90% M2 - >=75-90% M3 - >=50-75% , M4 >=25-50% , M5 <25% | No. of valid defects reported at a stage (e.g. UAT)/ Total no. of defects reported in the current stage and previous stage (UAT & QA)
| Sprint Name: <<Percentage Value>> Escaped Defects: <<Value>> Total Defects : <<Value>> |
| Definition
Rep to On Hover -
Remarks -
|
| ||||||||||||||||||
Defect Removal Efficiencygives a measure of efficiency of the development team in closing defects raised because of new functionalities within the iteration | Line Graph
| Agile Project Management
| M1 <25% M2 >=25-50% , M3 - >=50-75% , M4 - >=75-90% , M5 - >=90% | No. of defects tagged to stories in the iteration that are fixed/ Total no. of defects tagged to stories in a iteration
| Sprint Name: <<Percentage Value>> Closed Defects: <<Value>> Total Defects : <<Value>> | Remarks -
| ||||||||||||||||||||
Defect Rejection Ratemeasures the effectiveness of testing team | Line Graph
| Agile Project Management
| M1 - >=75% , M2 - >=50-75% , M3 - >=30-50% , M4 >=10-30% , M5 <10% | No. of defects tagged to stories in the iteration that are rejected / Total no. of defects tagged to stories in a iteration
| Sprint Name: <<Percentage Value>> Rejected Defects: <<Value>> Total Defects : <<Value>> |
Definition
Rep to On Hover -
Status | | |||||||||||||||||||
| colour | Green | title | DONE||||||||||||||||||||||||
Defect Count by Prioritymeasures number of defects for each priority defined in a project | Line chart
| Agile Project Management
| No Maturity Trend | No. of defects linked to stories in the iteration grouped by priority
| Sprint Name: P1: <<Value>> P2: <<Value>> P3: <<Value>> P4: <<Value>> | Remarks - | Status | | |||||||||||||||||||
| colour | Green | title | DONE
| Status | ||||
|---|---|---|---|---|
|
Rep to On Hover -
| Status | ||||
|---|---|---|---|---|
|
Total defects is the count of all linked defects to the closed stories
Defect Count By RCA
measures number of defects along with the root cause of defects
Line chart
Y- Axis - Count
X- Axis - Sprints
Aggregation Method - Sum
Positive trend indication - Decrease
Agile Project Management
Jira
Azure Boards
No Maturity Trend
No. of defects linked to stories in the iteration grouped by Root Cause
It is calculated as ‘Count’. Lower the count, better is the ‘Quality’
Sprint Name:
RCA1: <<Value>>
RCA2: <<Value>>
RCA3: <<Value>>
RCA4: <<Value>>
Stories are considered based on the field ‘Defect Count - Issue Types with Linked Defect--> Issue type mappings.
Defects that have status defined as per ‘Defect Rejection status’ or ‘Resolution type for rejection’ under Workflow status mapping are excluded from calculation
Total defects is the count of all linked defects to the closed stories
Definition
| Status | ||||
|---|---|---|---|---|
|
Rep to On Hover -
| Status | ||||
|---|---|---|---|---|
|
from calculation
Total defects is the count of all linked defects to the closed stories
Regression Automation Coverage
measures progress of automation of regression test suite in terms of percentage automated out of total regression testcases
Line Graph
Y- Axis - Percentage
X- Axis - Sprints
Aggregation Method - Average
Positive trend indication - Increase
Agile Project Management
Zephyr
Jira Test
M1 - <20%
M2 - 20-40%
M3 - 40-60%
M4 - 60% - 80% ,
M5 - > 80%
No. of regression test cases automated till the end of iteration/Total no. of regression test cases available till the end of iteration
It is calculated as a ‘Percentage’. Higher the percentage, better is the ‘Quality’
Maturity of the KPI is calculated based on the average of last 5 values that corresponds with the maturity scale
Sprint Name: <<Percentage Value>>
Regression test cases automated: <<Value>>
Total Regression test cases: <<Value>>
Following tools can be set up to get data on Regression Automation Coverage kpi
Zephyr (Server/Cloud) with Jira setup
Jira Test (Zephyr squad/X-ray) with Jira setup
Total regression testcases and automated testcases will be fetched based on the configured field values.
Testcases may or may not be linked to stories.
Definition
| Status | ||||
|---|---|---|---|---|
|
Rep to On Hover -
| Status | ||||
|---|---|---|---|---|
|
Remarks -
| Status | ||||
|---|---|---|---|---|
|
based on the configured field values.
Testcases may or may not be linked to stories.
In-Sprint Automation Coverage
measures progress of automation of test cases linked to stories within the iteration
Line Graph
Y- Axis - Percentage
X- Axis - Sprints
Aggregation Method - Average
Positive trend indication - Increase
Agile Project Management
Zephyr
Jira Test
M1 - <20%
M2 - 20-40%
M3 - 40-60%
M4 - 60% - 80% ,
M5 - > 80%
No. of test cases linked to stories in iteration that are automated/Total test cases linked to stories in iteration
It is calculated as a ‘Percentage’. Higher the percentage, better is the ‘Quality’
Maturity of the KPI is calculated based on the average of last 5 values that corresponds with the maturity scale
Sprint Name: <<Percentage Value>>
Linked test cases automated: <<Value>>
Total linked test cases: <<Value>>
Following tools can be set up to get data on Regression Automation Coverage kpi
Zephyr (Server/Cloud) with Jira setup
Jira Test (Zephyr squad/X-ray) with Jira setup
Total test cases will be fetched based on the testcases of linked stories which falls under the iteration and automated test cases will be calculated based on configured field values.
Definition
| Status | ||||
|---|---|---|---|---|
|
Rep to On Hover -
| Status | ||||
|---|---|---|---|---|
|
Remarks -
| Status | ||||
|---|---|---|---|---|
|
and automated test cases will be calculated based on configured field values.
Test Execution and pass percentage
measures the percentage of test cases that have been executed out of the total test cases with an iteration & the percentage that have passed.
Line Graph + Column chart
Y- Axis - Percentage
X- Axis - Sprints
Aggregation Method - Average
Positive trend indication - Increase
Jira
KnowHOW
M1 - <20%
M2 - 20-40%
M3 - 40-60%
M4 - 60% - 80%
M5 - > 80%
No. of executed test cases out of total test cases in the latest execution of a sprint
No. of pass test cases out of executed test cases in the latest execution of a sprint
It is calculated in ‘Percentage’. Higher the percentage, better is the ‘Quality’
Maturity of the KPI is calculated based on the average of last 5 values that corresponds with the maturity scale
Sprint Name: <<Percentage Value>>
Test Cycle:
Executed test cases: <<Value>>
Pass test cases: <<Value>>
The information of total, executed and pass test cases needs to be uploaded in KnowHOW (Settings--> Upload data) for every iteration
The maturity value defined is for the pass percentage
Definition
| Status | ||||
|---|---|---|---|---|
|
Rep to On Hover -
| Status | ||||
|---|---|---|---|---|
|
Unit Test Coverage
is a measurement of the amount of code that is run by unit tests - either lines, branches, or methods
Line Graph with single-select filter
Y- Axis - Percentage
X- Axis - Weeks
Aggregation Method - Average
Positive trend indication - Increase
Code Quality
Sonarqube
Sonar Cloud
M1 - <20%
M2 - 20-40%
M3 - 40-60%
M4 - 60% - 80%
M5 - > 80%
The calculation is done directly in Sonarqube
It is calculated in ‘Percentage’. Higher the percentage, better is the ‘Quality’
Maturity of the KPI is calculated based on the latest value
Date Range: << Percentage Value>>
Multiple branches can be configured for the same project
Sonar Violations
highlight count of issues that exist every time a piece of code breaks a coding rule. The set of coding rules is defined through the associated Quality profile for each programming language in the project.
Line chart
Y- Axis - Count
X- Axis - Weeks
Aggregation Method - Sum
Positive trend indication - Decrease
Code Quality
Sonarqube
Sonar Cloud
No Maturity Trend
The calculation is done directly in Sonarqube.
Issues are categorized in 3 types: Bug, Vulnerability and Code Smells
All these issues are categorized into 5 type of severity: Blocker, Critical, Major, Minor, Info
Date Range:
Blocker: <<Value>>
Critical: <<Value>>
Major: <<Value>>
Minor: <<Value>>
Info: <<Value>>
Multiple branches can be configured for the same project
Sonar Tech Debt
explains the estimated time required to fix all Maintainability Issues/ code smells
Line Graph
Y- Axis - Days
X- Axis - Weeks
Aggregation Method - Sum
Positive trend indication - Decrease
Code Quality
Sonarqube
Sonar Cloud
M1: >100 days
M2: 50-100 days
M3: 30-50 days,
M4: 10-30 days,
M5: <10 days
It is calculated as effort to fix all Code Smells. The effort is calculated in minutes and converted to days by assuming 1 Day=8 Hours
It is calculated in ‘Days’. Lower the days, better is the ‘Quality’
Maturity of the KPI is calculated based on the latest value
Date Range:<<Days>>
Multiple branches can be configured for the same project
Change Failure Rate
measures the proportion of builds (changes) that have failed for whatever reason over a given period of time
Line Graph with single-select filter
Y- Axis - Percentage
X- Axis - Weeks
Aggregation Method - Average
Positive trend indication - Decrease
Build
Jenkins
Bamboo
Azure Pipelines
Teamcity
M1: >50%
M2: 30-50%,
M3: 20-30%,
M4: 10-20%,
M5: <10%
Total no. of failed Builds/Total number of Builds
It is calculated as a ‘Percentage’. Lower the percentage, better is the ‘Quality’
Maturity of the KPI is calculated based on the average of last 5 values that corresponds with the maturity scale
*If the KPI data is not available for last 5 weeks, the Maturity level will not be shown
Date Range: <<Percentage>>
Total number of Changes: <<Value>>
Failed Changes: <<Value>>
Multiple jobs can be configured for the same project
Created vs Resolved
gives an idea about the speed at which the team closes the raised defects.
Line Graph + Column chart
Y- Axis - Count
X- Axis - Sprints
Aggregation Method - Sum
Positive trend indication - Increase
Agile Project Management
Jira
Azure
No Maturity
Created defects consider all defects which are initially tagged to the Sprint or are added to the sprint in between
Closed defects are the completed defects i the sprint out of the created count