Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages


home | help
Test::Group(3)	      User Contributed Perl Documentation	Test::Group(3)

       Test::Group - Group together related tests in a test suite

       Test::Group version 0.20


	   use Test::More no_plan => 1;
	   use Test::Group;

	   test	"hammering the server" => sub {
	       for(1..1000) {
	   }; #	Don't forget the semicolon here!

       Failed subtests are displayed before the	result of the test they	belong
       to.  For	instance,

	   use Test::More no_plan => 1;
	   use Test::Group;

	   test	"this test group will fail", sub {
	       ok 1, "sub test blah";
	       is "foo", "bar",	"I so wish they	were the same...";
	       ok 1;
	       like   "blah blah blah",	qr/bla/;

       produces	something like this:

	   #   Failed test 'I so wish they were	the same...'
	   #   in at line 6.
	   #	      got: 'foo'
	   #	 expected: 'bar'
	   not ok 1 - this test	group will fail
	   #   Failed test 'this test group will fail'
	   #   in at line 9.

       Exceptions in tests are not fatal:

	   test	"this test will	fail but the suite will	proceed", sub {

       Test::More style	TODO support:

	   test	"a test	with TODO in the name is marked	TODO" => sub {
		 pass("this part is done");
		 fail("but I'm not finished with this one yet");

	     local $TODO = "Test::More's good old method also works";
	     test "this	test is	not finished yet" => sub {


	   # Don't catch exceptions raised in test groups later	on

	   # log caught	exceptions in /tmp/log

	   # skip the next group of test
	   skip_next_test "network not available" if (!	Network->available());
	   test	"bla", sub {
	       my $ftp = Net::FTP->new("");
	       # ...

	   begin_skipping_tests	"reason";

	   test	"this test will	not run" => sub	{
	       # ...


	   # from now on, skip all tests whose names do	not match /bla/
	   test_only qr/bla/;

       Fed up with counting tests to discover what went	wrong in your last
       test run?  Tired	of squinting at	your test source to find out where on
       earth the faulty	test predicate is called, and what it is supposed to
       check for?  Then	this module is for you!

       Test::Group allows for grouping together	related	tests in a standard
       Test::More-style	script.	(If you	are not	already	familiar with
       Test::More, now would be	the time to go take a look.)  Test::Group
       provides	a bunch	of maintainability and scalability advantages to large
       test suites:

       o   related tests can be	grouped	and given a name. The intent of	the
	   test	author is therefore made explicit with much less effort	than
	   would be needed to name all the individual tests;

       o   the test output is much shorter and more readable: only failed
	   subtests show a diagnostic, while test groups with no problems
	   inside produce a single friendly "ok" line;

       o   no more tedious test	counting: running an arbitrarily large or
	   variable number of tests (e.g. in loops) is now hassle-free and
	   doesn't clutter the test output.

       Authors of Test::* modules may also find	Test::Group of interest,
       because it allows for composing several Test::More predicates into a
       single one (see "Reflexivity").

   Blocking Exceptions
       By default, calls to "die" in perlfunc and other	exceptions from	within
       a test group cause it to	fail and terminates execution of the group,
       but does	not terminate whole script.  This relieves the programmer from
       having to worry about code that may throw in tests.

       This behavior can be disabled totally using "dont_catch_exceptions".
       Exceptions can also be trapped as usual using "eval" in perlfunc	or
       otherwise from inside a group, in which case the	test code of course
       has full	control	on what	to do next (this is how	one should test	error
       management, by the way).

       When Test::Group	is set to block	errors (the default setting, see also
       "catch_exceptions"), the	error messages are displayed as	part of	the
       test name, which	some may not find very readable.  Therefore, one can
       use a "logfile" instead.

   Skipping Groups
       Test::Group can skip single test	groups or a range of them (consecutive
       or matched by a regex), which helps shortening the debug	cycle even
       more in test-driven programming.	 When a	test group is skipped, the
       code within it is simply	not executed, and the test is marked as
       skipped wrt Test::Builder.  See "skip_next_test", "skip_next_tests",
       "begin_skipping_tests", "end_skipping_tests" and	"test_only" for

       Test groups integrate with Test::Builder	by acting as a single big
       test; therefore,	Test::Group is fully reflexive.	 A particularly
       attractive consequence is that constructing new Test::More predicates
       is straightforward with Test::Group.  For example,

	   use Test::Builder;
	   use Test::Group;

	   sub foobar_ok {
	       my ($text, $name) = @_;
	       $name ||= "foobar_ok";
	       local $Test::Builder::Level = $Test::Builder::Level + 1;
	       test $name => sub {
		  local	$Test::Group::InPredicate = 1;
		  like($text, qr/foo/, "foo ok");
		  like($text, qr/bar/, "bar ok");

       defines a new test predicate foobar_ok that will	DWIM regardless	of the
       caller's	testing	style: for "classical" Test::Simple or Test::More
       users, foobar_ok	will act as just another *_ok predicate	(in
       particular, it always counts for	a single test, honors "TODO: BLOCK" in
       Test::More constructs, etc); and	of course, users of Test::Group	can
       freely call foobar_ok from within a group.

       Adding 1	to $Test::Builder::Level causes	the location of	the call to
       foobar_ok() to be shown if a test fails,	see Test::Builder.

       Setting $Test::Group::InPredicate to a true value prevents the location
       of individual failing tests within test groups from being shown.

   TODO	Tests
       As shown	in "SYNOPSIS", Test::More's concept of TODO tests is supported
       by Test::Group: a group is in TODO state	if the $TODO variable is set
       by the time it starts, or if the	test name contains the word "TODO".
       Note, however, that setting $TODO from inside the test group (that is,
       after the group starts) will not	do what	you mean:

	  test "something" => sub {
	      local $TODO = "this test does not	work yet";
	      pass;					    # GOTCHA!

       Here "pass" is an unexpected success, and therefore the whole test
       group will report a TODO	success	despite	the test not actually being a
       success (that is, it would also be defective if one were	to comment out
       the "local $TODO" line).	 This semantics, on the	other hand, DWIMs for
       marking a portion of the	test group as TODO:

	  test "something" => sub {
		 local $TODO = "this part does not work	yet";

       Finally,	there is a subtle gotcha to be aware of	when setting $TODO
       outside a test group (that's the	second one, so maybe you should	not do
       that to begin with).  In	this case, the value of	$TODO is set to	undef
       inside the group.  In other words, this test (similar to	the one	to be
       found in	"SYNOPSIS") will succeed as expected:

	     local $TODO = "not	quite done yet";
	     test "foo"	=> sub {
		 pass;		    # NOT an unexpected	success, as
				    # this is simply a subtest of the whole
				    # test "foo", which	will fail.

       As seen briefly in "SYNOPSIS", only top-level test groups (and toplevel
       Test::More tests	if any)	produce	a single "ok" or "not ok" summary
       line.  Failed sub-tests produce non-scoring comment messages (prefixed
       with "#"); successful sub-tests are silent.  This is different from,
       and predates, the "subtest" in Test::More functionality.

       However,	if you enable the experimental "use_subtest" feature then
       Test::Group will	use the	same underlying	mechanism as "subtest" in
       Test::More and produce very similar output.

       A simple	plugin interface allows	module authors to write	extensions to
       Test::Group.  See Test::Group::Extending	for details.

       The following extensions	are distributed	with Test::Group:

       Test::Group::Plan, Test::Group::NoWarnings

       All functions below are intended	to be called from the test script.
       They are	all exported by	default.

       test ($name, $groupsub)

       Executes	$groupsub, which must be a reference to	a subroutine, in a
       controlled environment and groups the results of	all
       Test::Builder-style subtests launched inside into a single call to "ok"
       in Test::Builder, regardless of their number.  If the test group	is to
       be skipped (as discussed	in "Skipping Groups"), calls "skip" in
       Test::Builder once instead.

       In case the test	group is not skipped, the first	parameter to "ok" in
       Test::Builder and the value of the TODO string during same (see "TODO:
       BLOCK" in Test::More) are determined according to the following

       1.  if the test group terminates	by throwing an exception, or
	   terminates normally but without calling any subtest,	it fails.

       2.  otherwise, if any subtest failed outside of a TODO block, the group

       3.  otherwise, if any subtest succeeds inside of	a TODO block, the
	   group is flagged as an unexpected success.

       4.  otherwise, if any subtest fails inside of a TODO block, the group
	   results in a	TODO (excused) failure.

       5.  otherwise, the test group managed to	avert all hazards and is a
	   straight success (tada!!).

       If any sub-tests	failed in $groupsub, diagnostics will be propagated
       using "diag" in Test::Builder as	usual.

       The return value	of test	is 1 if	the test group is a success (including
       a TODO unexpected success), 0 if	it is a	failure	(including a TODO
       excused failure), and undef if the test group was skipped.

       skip_next_tests ($number)

       skip_next_tests ($number, $reason)

       Skips the $number following groups of tests with	reason $reason.	 Dies
       if we are currently skipping tests already.

       skip_next_test ()

       skip_next_test ($reason)

       Equivalent to:

	   skip_next_tests 1;
	   skip_next_tests 1, $reason;

       begin_skipping_tests ()

	   begin_skipping_tests	"reason";

       Skips all subsequent groups of tests until blocked by

       end_skipping_tests ()

       Cancels the effect of "begin_skipping_tests". Has no effect if we are
       not currently skipping tests.

       test_only ()

	   test_only "bla()", "reason";
	   test_only qr/^bla/;
	   test_only sub { /bla/ };

       Skip all	groups of tests	whose name does	not match the criteria.	 The
       criteria	can be a plain string, a regular expression or a function.


       Resets to normal	behavior.

       The following function relates to the plugin interface. It is not
       exported	by default.  See Test::Group::Extending	for details.

       next_test_plugin	($plugin)

       Installs	a plugin for the next test group. $plugin must be a subroutine

       A handful of class methods are available	to tweak the behavior of this
       module on a global basis. They are to be	invoked	like this:


   verbose ($level)
       Sets verbosity level to $level, where 0 means quietest.

       At level	1 and above there is a diagnostic line for the start of	each
       test group.

       At level	2 there	is a diagnostic	line showing the result	of each
       subtest within top-level	test groups. At	level 3, the subtests of test
       groups nested within top	level test groups also get diagnostic lines,
       and so on.

       The default verbosity level is 0, or the	value of the
       "PERL_TEST_GROUP_VERBOSE" environment variable if it is set.

   catch_exceptions ()
       Causes exceptions thrown	from within the	sub reference passed to	"test"
       to be blocked; in this case, the	test currently running will fail but
       the suite will proceed. This is the default behavior.

       Note that catch_exceptions only deals with exceptions arising inside
       test blocks; those thrown by surrounding	code (if any) still cause the
       test script to terminate	as usual unless	other appropriate steps	are

   dont_catch_exceptions ()
       Reverses	the effect of "catch_exceptions", and causes exceptions	thrown
       from a "test" sub reference to be fatal to the whole suite.  This only
       takes effect for	test subs that run after dont_catch_exceptions()
       returns;	in other words this is not a whole-script pragma.

   logfile ($classstate_logfile)
       Sets the	log file for caught exceptions to $classstate_logfile.	From
       this point on, all exceptions thrown from within	a text group (assuming
       they are	caught,	see "catch_exceptions")	will be	written	to
       $classstate_logfile instead of being passed on to "diag"	in Test::More.
       This is very convenient with exceptions with a huge text	representation
       (say an instance	of Error containing a stack trace).

   use_subtest ()
       This feature is experimental.

       Causes Test::Group to use Test::Builder's subtest() feature as the test
       aggregation method, rather than doing black magic behind	the scenes.

       It is a fatal error to call use_subtest() if Test::Builder is too old
       to support subtests.  To	use subtests if	they are available but fall
       back to normal operation	if they	are not, you can do:

	 eval {	Test::Group->use_subtest };

       Test::Group's exception handling	mechanism is bypassed under
       use_subtest(), since Test::Builder::subtest() has its own exception
       handling	system.

   no_subtest ()
       Turns off use_subtest.

       This class uses a somewhat unhealthy dose of black magic	to take	over
       control from Test::Builder when running inside a	"test" group sub.
       While the temporary re-blessing trick used therein is thought to	be
       very robust, it is not very elegant.

       The experimental	use_subtest() feature allows you to avoid the black
       magic if	your Test::Builder is recent enough to support subtests.

       Test::Simple, Test::More, Test::Builder,	and friends

       The "perl-qa" project, <>.

   Similar modules on CPAN
       Test::Class can be used to turn a test suite into a full-fledged	object
       class of	its own, in xUnit style.  It also happens to support a similar
       form of test grouping using the ":Test(no_plan)"	or ":Tests"
       attributes.  Switching over to Test::Class will make a test suite more
       rugged and provide a number of advantages, but it will also dilute the
       "quick-and-dirty" aspect	of .t files somewhat. This may or may not be
       what you	want: for example, the author of this module enjoys
       programming most	when writing tests, because the	most infamous Perl
       hacks are par for the course then :-).  Anyway TIMTOWTDI, and
       Test::Group is a	way to reap some of the	benefits of Test::Class	(e.g.
       running only part of the	test suite) without changing one's programming
       style too much.

       Nick Cleaton <>

       Dominique Quatravaux <>

       Nicolas M. Thiery <>

       Copyright (C) 2004 by IDEALX <>

       Copyright (c) 2009 by Nick Cleaton and Dominique	Quatravaux

       This library is free software; you can redistribute it and/or modify it
       under the same terms as Perl itself, either Perl	version	5.8.1 or, at
       your option, any	later version of Perl 5	you may	have available.

perl v5.32.0			  2020-08-11			Test::Group(3)


Want to link to this manual page? Use this URL:

home | help