Application Coverage Is the Most Powerful Metric You’re Not Measuring

Code coverage is an important, yet often neglected, step in an overall software quality process that enables you to begin understanding where risk is lurking in your application. Furthermore, the methods by which coverage is measured matter—different activities measure different dimensions of the application.

Enter application coverage, which expresses the degree to which the application’s source code is exercised by all testing practices. Application coverage provides extremely powerful insight into risk because it exposes untested or under-tested parts of the application. Before getting into the details of application coverage, let’s discuss how code coverage is usually measured.

What’s Wrong With How Teams Normally Measure Coverage?

Dev/test teams typically lean on unit testing as the primary vehicle for driving coverage. Unit testing also provides many other valuable benefits, such as:

• Instant and automatic proof that the code works anytime, anywhere.
• Confidence to refactor code, resulting in improved design and better maintainability.
• Almost instant feedback if something breaks.
• The ability to validate the code before checking it in because tests can run locally and independently of the continuous integration infrastructure.

Let’s make it clear that we’re ardent proponents of unit testing. However, while unit testing is valuable for ensuring the correctness of individual units of code, a combination of testing approaches, including integration testing, functional testing, performance testing, and even manual testing (yes, manual testing), is essential to comprehensively assess the overall quality and reliability of a software application.

It’s essential to recognize that evaluating coverage during functional and manual test executions holds equal importance. This approach allows you to measure the comprehensiveness and quality of your functional and manual test executions.

Why Teams Should Add Application Coverage to the Dev/Test Mix

A critical metric for assessing release readiness is application coverage. Application coverage is the aggregation and correlation of code coverage from various testing techniques. By aggregating code coverage from all testing practices, organizations can have better visibility into risky areas of the application.

No Time for Testing? No Problem

In addition to enabling more accurate quality and coverage metrics, application coverage can help organizations focus testing efforts when time constraints limit their ability to run the full suite of manual regression tests. In exploratory testing, for example, testers know that the code around certain functionality has changed and quickly attempt to identify unexpected behavior by performing manual testing around the changed functionality.

Capturing coverage data on the running application during exploratory testing potentially increases the value of this approach because it enables a better understanding of what code the tester touched.

A development testing platform such as Parasoft DTP can reduce the dependency on arbitrary exploratory testing by providing teams with a better understanding of what changed, what areas are potentially impacted, and what needs to be retested. Exploratory testing then becomes an additional layer of testing used to fill in any gaps.

No Single Metric or Activity Is a Defect Prevention Cure-All

It’s important to note that application coverage alone is not a credible measure of application quality. Coverage data should also be measured within the context of other software quality metrics, such as static analysis violations, unit test results, and the like, to develop a comprehensive understanding of risk from all angles.

The screenshot below shows a view of Parasoft’s Development Testing Platform (DTP) with coverage data, unit test results, functional testing results, static analysis violations, and other metrics. This level of granular detail helps managers understand the current state of risk in the application, as well as the impact of recent changes on quality metrics.

Get a Better View of Application Coverage

Teams get a better view of testing outcomes and application coverage with insightful reporting and analytics provided by platforms like DTP. It merges and correlates results across testing practices to provide continuous monitoring of testing outcomes. Teams can see what’s working and what isn’t.

Want to see DTP’s application coverage capabilities first-hand? Request a demo with one of our experts.