Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
SHTK_UNITTEST(3)       FreeBSD Library Functions Manual	      SHTK_UNITTEST(3)

NAME
     unittest -- Utilities to implement	test programs

LIBRARY
     shtk_import unittest

DESCRIPTION
     The unittest module provides a framework with which to implement test
     programs.

     A test program implemented	with the unittest library is made up of	a com-
     bination of standalone test cases and/or test cases within	test fixtures.
     The test program will exit	with 0 (success) if and	only if	all tests
     pass, and 1 (failure) if any single test fails.  Skips and	expected fail-
     ures are considered passes.

   Program entry point
     Test programs implemented using unittest must use the
     shtk_unittest_main(3) function as their entry point.  There are two mech-
     anisms to achieve this.

     The first is to end the test program with the following definition	of
     main:

	   main() { shtk_unittest_main "${@}"; }

     The second	is to tell shtk(1) at build time to use	shtk_unittest_main(3)
     as	the program's entry point:

	   $ shtk build	-m shtk_unittest_main module_test.sh

     In	general, prefer	the latter mechanism.

   Asserts vs. expects
     The unittest library provides a variety of	helper check functions.	 One
     such example is the `equal' check,	which allows the caller	to compare two
     values for	equality and fail the test case	with the appropriate error
     message if	the values differ.

     Each check	is offered in two flavors: an assert and an expect, which in
     the previous example means	we have	access to the two corresponding	func-
     tions assert_equal	and expect_equal.  Assert-style	checks are fatal: when
     the condition they	test for is not	met, the test case is immediately
     aborted; instead, expect-style checks are not fatal: they record the cur-
     rently-running test case as failed	but they let the test case continue
     execution.	 Assert-style checks should be used to validate	conditions
     that must hold true in order to run the test (e.g.	ensuring that the
     setup of the test case succeeds); expect-style checks should be used for
     all other cases.

   Scoping
     All functions provided by the unittest library are	prefixed by
     `shtk_unittest_' as you would expect from the coding practices of shtk.
     However, such long	prefix is inconvenient when writing test programs, as
     test programs are a very special case of scripts in which a domain-spe-
     cific language makes sense.

     Therefore,	for simplicity reasons,	test cases have	access to shortened
     names of the unittest functions without the `shtk_unittest_' prefix.  In
     order words, a test case can simply call fail instead of having to	call
     shtk_unittest_fail.

   Standalone test cases
     Standalone	test cases are defined by top-level functions whose name ends
     with the `_test' suffix and are registered	as test	cases with the
     shtk_unittest_add_test(3) function.  Such test cases are run indepen-
     dently of each other, with	no common setup	nor teardown code among	them.

   Test	fixtures
     Test fixtures are collections of related test cases that may share	op-
     tional setup and teardown methods.	 Test fixtures are defined by top-
     level functions whose name	ends with the `_fixture' suffix	and are	regis-
     tered as fixtures with the	shtk_unittest_add_fixture(3) function.

     Within fixture fuctions, there may	be a setup and a teardown method, and
     there should be one or more test cases registered with
     shtk_unittest_add_test(3).	 Each test within the fixture is executed in-
     dependently of each other,	but the	setup and teardown methods (if any)
     are also executed with the	test.

   Test	environment
     The runtime engine	(usually kyua(1)) is responsible for executing the
     test program under	a controlled directory and with	a sanitized environ-
     ment.

     However, due to the nature	of shell-based test programs, unittest creates
     one extra directory for each test case, and such directory	becomes	the
     work directory of the test	case.  These directories are all created
     within the	directory in which the test program was	executed.  To ensure
     that any files created by the test	case remain within its own subdirec-
     tory as much as possible, the HOME	and TMPDIR variables are set to	point
     to	the individual test's work directory.

   One time setup and teardown
     Test programs may optionally define top-level one_time_setup and
     one_time_teardown.

     The code supplied in the one_time_setup function is executed exactly once
     at	the beginning of the test program and all state	set up by the function
     is	shared across all tests.

     The code supplied in the one_time_teardown	function is executed exactly
     once at the end of	the test program and should be used to clean up	any
     state prepared by one_time_setup.

     It	is important to	mention	that these one-time setup and teardown rou-
     tines run in the parent directory of the executed test cases.  In other
     words: if the setup routine creates any file to be	shared across all
     tests, the	tests will have	access to such files by	looking	them up	in
     their parent directory.

   Test	results
     Any test case can complete	with one of the	following results:

     Expected failure (pass)
	   The test case detected a failure, but such failure is expected.
	   This	condition is often use to denote test cases that are either
	   incomplete or test cases that exercise a known bug.	See
	   shtk_unittest_set_expected_failure(3) for details.

     Failure
	   The test case explicitly failed or it unexpectedly terminated.  See
	   shtk_unittest_delayed_fail(3) and shtk_unittest_fail(3) for de-
	   tails.

     Pass  The test case completed successfully	without	reporting any errors.

     Skip (pass)
	   The test case determined that it cannot run through completion and
	   decided to abort early instead of reporting a false negative.  See
	   shtk_unittest_skip(3) for details.

EXAMPLES
     The most basic test program that can be written is	the following, which
     provides a	simple test case for cp(1)'s -f	flag:

	   shtk_import unittest

	   shtk_unittest_add_test cp_f_forces_override
	   cp_f_forces_override_test() {
	       echo "first" >first
	       echo "second" >second
	       chmod 555 second

	       # First make sure that cp without -f fails.
	       assert_command -s exit:1	-e ignore cp first second

	       # Now run a second attempt with -f and verify that the command
	       # exits successfully and	it is silent.
	       assert_command cp -f first second

	       cmp -s first second || fail "source and destination do not match"
	   }

	   main() { shtk_unittest_main "${@}"; }

     A more complex test program can use fixtures to group related test	cases
     and to provide common setup and teardown code for each of them:

	   shtk_import unittest

	   shtk_unittest_add_fixture cp
	       setup() {
		   # Create common files used by all test cases.  Note that this runs
		   # once per test case, so state is not shared	among them.
		   echo	"first"	>first
		   echo	"second" >second
	       }

	       teardown() {
		   # Common cleanup to be executed at the end of the test case,	no
		   # matter if it passes or fails.
		   rm -f first second
	       }

	       shtk_unittest_add_test override_existing_file
	       override_existing_file_test() {
		   assert_command cp first second
		   cmp -s first	second || fail "source and destination do not match"
	       }

	       shtk_unittest_add_test f_forces_override
	       f_forces_override_test()	{
		   chmod 555 second
		   assert_command -s exit:1 -e ignore cp first second
		   assert_command cp -f	first second
		   cmp -s first	second || fail "source and destination do not match"
	       }
	   }

	   main() { shtk_unittest_main "${@}"; }

     Lastly, the most complex test program is depicted here, which includes a
     combination of fixtures and test cases with one-time setup	and teardown
     routines:

	   shtk_import unittest

	   one_time_setup() {
	       ... initialization code shared by all tests in the program ...
	   }

	   one_time_teardown() {
	       ... clean up code shared	by all tests in	the program ...
	   }

	   shtk_unittest_add_fixture clients
	   clients_fixture() {
	       setup() {
		   ... initialization code shared by all tests in the fixture ...
	       }
	       teardown() {
		   ... cleanup code shared by all tests	in the fixture ...
	       }

	       shtk_unittest_add_test add
	       add_test() {
		   ... first test in the fixture ...
	       }

	       shtk_unittest_add_test modify
	       modify_test() {
		   ... second test in the fixture ...
		   fail	"And it	fails"
	       }
	   }

	   shtk_unittest_add_test initialization
	   initialization_test() {
	       ... standalone test not part of any fixture ...
	       skip "But cannot	run due	to some	unsatisfied condition"
	   }

	   # Either do this or,	preferably, pass -mshtk_unittest_main to
	   # "shtk build" when compiling the test program and don't define main.
	   main() { shtk_unittest_main "${@}"; }

SEE ALSO
     shtk(3), shtk_unittest_add_fixture(3), shtk_unittest_add_test(3),
     shtk_unittest_delayed_fail(3), shtk_unittest_fail(3),
     shtk_unittest_main(3), shtk_unittest_set_expected_failure(3),
     shtk_unittest_skip(3)

   Checks
     shtk_unittest_assert_command(3), shtk_unittest_assert_equal(3),
     shtk_unittest_assert_file(3), shtk_unittest_assert_not_equal(3),
     shtk_unittest_expect_command(3), shtk_unittest_expect_equal(3),
     shtk_unittest_expect_file(3), shtk_unittest_expect_not_equal(3)

HISTORY
     unittest first appeared in	shtk 1.6.

FreeBSD	13.0		       November	16, 2014		  FreeBSD 13.0

NAME | LIBRARY | DESCRIPTION | EXAMPLES | SEE ALSO | HISTORY

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=shtk_unittest&sektion=3&manpath=FreeBSD+13.0-RELEASE+and+Ports>

home | help