Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
Devel::Cover::TutorialUser Contributed Perl DocumentaDevel::Cover::Tutorial(3)

NAME
       Devel::Cover::Tutorial -	An introduction	to code	coverage

VERSION
       version 1.36

TUTORIAL
       Here's part of a	message	I sent to perl-qa about	code coverage metrics.

   1.0 Introduction
       It is wise to remember the following quote from Dijkstra, who said:

	 Testing never proves the absence of faults, it	only shows their presence.

       In particular, code coverage is just one	weapon in the software
       engineer's testing arsenal.

       Any discussion of code coverage metrics is hampered by the fact that
       many authors use	different terms	to describe the	same kind of coverage.
       Here, I shall provide only a brief introduction to some of the most
       common metrics.

   2.0 Metrics
   2.1 Statement coverage
       This is the most	basic form of code coverage.  A	statement is covered
       if it is	executed.  Note	that statement != line of code.	 Multiple
       statements on a single line can confuse issues -	the reporting if
       nothing else.

       Where there are sequences of statements without branches	it is not
       necessary to count the execution	of every statement, just one will
       suffice,	but people often like the count	of every line to be reported,
       especially in summary statistics.  However it is	not clear to me	that
       this is actually	useful.

       This type of coverage is	fairly weak in that even with 100% statement
       coverage	there may still	be serious problems in a program which could
       be discovered through other types of metric.

       It can be quite difficult to achieve 100% statement coverage.  There
       may be sections of code designed	to deal	with error conditions, or
       rarely occurring	events such as a signal	received during	a certain
       section of code.	 There may also	be code	that should never be executed:

	 if ($param > 20) {
	   die "This should never happen!";
	 }

       It can be useful	to mark	such code in some way and flag an error	if it
       is executed.

       Statement coverage, or something	very similar, can be called statement
       execution, line,	block, basic block or segment coverage.	 I tend	to
       favour block coverage which does	not attempt to extend its results to
       each statement.

   2.2 Branch coverage
       The goal	of branch coverage is to ensure	that whenever a	program	can
       jump, it	jumps to all possible destinations.  The most simple example
       is a complete if	statement:

	 if ($x) {
	   print "a";
	 } else	{
	   print "b";
	 }

       In such a simple	example	statement coverage is as powerful, but branch
       coverage	should also allow for the case where the else part is missing:

	 if ($x) {
	   print "a";
	 }

       Full coverage is	only achieved here if $x is true on one	occasion and
       false on	another.

       100% branch coverage implies 100% statement coverage.

       Branch coverage is also called decision or all edges coverage.

   2.3 Path coverage
       There are classes of errors that	branch coverage	cannot detect, such
       as:

	 $h = undef;
	 if ($x) {
	   $h =	{ a => 1 };
	 } if ($y) {
	   print $h->{a};
	 }

       100% branch coverage can	be achieved by setting ($x, $y)	to (1, 1) and
       then to (0, 0).	But if we have (0, 1) then things go bang.

       The purpose of path coverage is to ensure that all paths	through	the
       program are taken.  In any reasonably sized program there will be an
       enormous	number of paths	through	the program and	so in practice the
       paths can be limited to a single	subroutine, if the subroutine is not
       too big,	or simply to two consecutive branches.

       In the above example there are four paths which correspond to the truth
       table for $x and	$y.  To	achieve	100% path coverage they	must all be
       taken.  Note that missing elses count as	paths.

       In some cases it	may be impossible to achieve 100% path coverage:

	 a if $x;
	 b;
	 c if $x;

       50% path	coverage is the	best you can get here.

       Loops also contribute to	paths, and pose	their own problems which I'll
       ignore for now.

       100% path coverage implies 100% branch coverage.

       Path coverage and some of its close cousins, are	also known as
       predicate, basis	path and LCSAJ (Linear Code Sequence and Jump)
       coverage.

   2.4 Expression coverage
       When a boolean expression is evaluated it can be	useful to ensure that
       all the terms in	the expression are exercised.  For example:

	 a if $x || $y

       The expression should be	exercised with ($x, $y)	set to (0, 0)
       (required for branch coverage), (0, 1) and (1, 0) (to ensure that $x
       and $y are independent) and possibly with (1, 1).

       Expression coverage gets	complicated, and difficult to achieve, as the
       expression gets complicated.

       Expressions which are not directly a part of a branching	construct
       should also be covered:

	 $z = $x || $y;
	 a if $z;

       Expression coverage is also known as condition, condition-decision and
       multiple	decision coverage.

   3.0 Other considerations
       In order	to get people to actually use code coverage it needs to	be
       simple to use.  It should also be simple	to understand the results and
       to rectify any problems thrown up.  Finally, if the overhead is too
       great it	won't get used either.

       So there's a basic tutorial on code coverage, or	at least my version of
       it.  Typing a few of these terms	into google will probably provide a
       basis for future	research.

LICENCE
       Copyright 2001-2019, Paul Johnson (paul@pjcj.net)

       This software is	free.  It is licensed under the	same terms as Perl
       itself.

       The latest version of this software should be available from my
       homepage: http://www.pjcj.net

perl v5.32.1			  2020-05-19	     Devel::Cover::Tutorial(3)

NAME | VERSION | TUTORIAL | LICENCE

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=Devel::Cover::Tutorial&sektion=3&manpath=FreeBSD+13.0-RELEASE+and+Ports>

home | help