Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
TESTS(7)	   FreeBSD Miscellaneous Information Manual	      TESTS(7)

NAME
     tests -- introduction to the FreeBSD Test Suite

DESCRIPTION
     The FreeBSD Test Suite provides a collection of automated tests for two
     major purposes.  On one hand, the test suite aids developers to detect
     bugs and regressions when they modify the source tree.  On	the other
     hand, it allows end users (and, in	particular, system administrators) to
     verify that fresh installations of	the FreeBSD operating system behave
     correctly on their	hardware platform and also to ensure that the system
     does not suffer from regressions during regular operation and mainte-
     nance.

     The FreeBSD Test Suite can	be found in the	/usr/tests hierarchy.

     This manual page describes	how to run the test suite and how to configure
     some of its optional features.

   Installing the test suite
     The test suite is installed by default as of FreeBSD 11.0-RELEASE.

     If	the /usr/tests directory is missing, then you will have	to enable the
     build of the test suite, rebuild your system and install the results.
     You can do	so by setting `WITH_TESTS=yes' in your /etc/src.conf file (see
     src.conf(5) for details) and rebuilding the system	as described in
     build(7).

   When	to run the tests?
     Before diving into	the details of how to run the test suite, here are
     some scenarios in which you should	run it:

	   +o   After a fresh installation of FreeBSD to	ensure that the	system
	       works correctly on your hardware	platform.

	   +o   After an	upgrade	of FreeBSD to a	different version to ensure
	       that the	new code works well on your hardware platform and that
	       the upgrade did not introduce regressions in your configura-
	       tion.

	   +o   After modifying the source tree to detect any new bugs and/or
	       regressions.

	   +o   Periodically, maybe from	a cron(8) job, to ensure that any
	       changes to the system (such as the installation of third-party
	       packages	or manual modifications	to configuration files)	do not
	       introduce unexpected failures.

   Running the tests
     First, you	will need to install the `devel/kyua' package from ports(7).
     Then use the following command to run the whole test suite:

	   $ kyua test -k /usr/tests/Kyuafile

     The above will iterate through all	test programs in /usr/tests recur-
     sively, execute them, store their results and debugging data in Kyua's
     database (by default in ~/.kyua/store.db),	and print a summary of the
     results.  This summary includes a brief count of all total	tests run and
     how many of them failed.

     It	is possible to restrict	which tests to run by providing	their names in
     the command line.	For example, this would	execute	the tests for the
     cp(1) and cut(1) utilities:

	   $ kyua test -k /usr/tests/Kyuafile bin/cp usr.bin/cut

   Obtaining reports of	the tests execution
     Additional	information about the test results can be retrieved by using
     Kyua's various reporting commands.	 For example, the following would
     print a plain-text	report of the executed tests and show which ones
     failed:

	   $ kyua report

     This example would	generate an HTML report	ready to be published on a web
     server:

	   $ kyua report-html --output ~/public_html/tests

     For further details on the	command-line interface of Kyua,	please refer
     to	its manual page	kyua(1).

   Configuring the tests
     Some test cases in	the FreeBSD Test Suite require manual configuration by
     the administrator before they can be run.	Unless certain properties are
     defined, the tests	that require them will be skipped.

     Test suites are configured	by defining their configuration	variables in
     /usr/local/etc/kyua/kyua.conf.  The format	of this	file is	detailed in
     kyua.conf(5).

     The following configuration variables are available in the	FreeBSD	Test
     Suite:

     allow_devfs_side_effects	If defined, enables tests that may destroy and
				recreate semipermanent device nodes, like disk
				devices.  Without this variable, tests may
				still create and destroy devices nodes that
				are normally transient,	like /dev/tap* and
				/dev/pts*, as long as they clean them up
				afterwards.  However, tests that require this
				variable have a	relaxed	cleanup	requirement;
				they must recreate any devices that they
				destroyed, but not necessarily with the	same
				devnames.

     allow_sysctl_side_effects	Enables	tests that change globally significant
				sysctl(8) variables.  The tests	will undo any
				changes	in their cleanup phases.

     disks			Must be	set to a space delimited list of disk
				device nodes.  Tests that need destructive
				access to disks	must use these devices.	 Tests
				are not	required to preserve any data present
				on these disks.

     fibs			Must be	set to a space delimited list of FIBs
				(routing tables).  Tests that need to modify a
				routing	table may use any of these.  Tests
				will cleanup any new routes that they create.

   What	to do if something fails?
     If	there is any failure during the	execution of the test suite, please
     consider reporting	it to the FreeBSD developers so	that the failure can
     be	analyzed and fixed.  To	do so, either send a message to	the appropri-
     ate mailing list or file a	problem	report.	 For more details please refer
     to:
	   +o   FreeBSD Mailing Lists:
		     https://lists.freebsd.org/
	   +o   Problem Reporting:
		     https://www.freebsd.org/support.html

FILES
     /usr/local/etc/kyua/kyua.conf   System-wide configuration file for
				     kyua(1).
     ~/.kyua/kyua.conf		     User-specific configuration file for
				     kyua(1); overrides	the system file.
     ~/.kyua/store.db		     Default result database used by Kyua.
     /usr/tests/		     Location of the FreeBSD Test Suite.
     /usr/tests/Kyuafile	     Top-level test suite definition file.

SEE ALSO
     kyua(1), build(7)

HISTORY
     The FreeBSD Test Suite first appeared in FreeBSD 10.1.

     The tests manual page first appeared in NetBSD 6.0	and was	later ported
     to	FreeBSD	10.1.

AUTHORS
     Julio Merino <jmmv@FreeBSD.org>

FreeBSD	Ports 11.2		August 21, 2017		    FreeBSD Ports 11.2

NAME | DESCRIPTION | FILES | SEE ALSO | HISTORY | AUTHORS

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=tests&sektion=7&manpath=FreeBSD+12.0-RELEASE+and+Ports>

home | help