Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
CHECKMK(1)							    CHECKMK(1)

NAME
       checkmk	-  Awk	script	for  generating	 C unit	tests for use with the
       Check unit testing framework.

SYNOPSIS
       checkmk [ clean_mode=1 ]	[ input-file ]

DESCRIPTION
       Generate	C-language source files	containing unit	tests for use with the
       Check  unit  testing  framework.	 The aim of this script	is to automate
       away some of the	typical	boilerplate one	must write when	writing	a test
       suite  using  Check:  specifically,  the	 instantiation	of an SRunner,
       Suite(s), and TCase(s), and the building	of relationships between these
       objects and the test functions.

       This  tool  is  intended	 to be used by those who are familiar with the
       Check unit testing framework. Familiarity with the  framework  will  be
       assumed throughout this manual.

       The  Check framework, along with	information regarding it, is available
       at http://check.sourceforge.net/	<URL:http://check.sourceforge.net/>.

       The input-file argument to checkmk uses a  simple,  C-preprocessor-like
       syntax  to  declare test	functions, and to describe their relationships
       to Suites and TCases in Check.  checkmk then uses this  information  to
       automatically  write  a main() function containing all of the necessary
       declarations, and whatever code is needed to run	the test  suites.  The
       final C-language	output is printed to checkmk's standard	output.

       Facilities  are provided	for the	insertion of user code into the	gener-
       ated main() function, to	provide	for the	use of logging,	test  fixtures
       or specialized exit values.

       While  it  is  possible	to omit	the input-file argument	to checkmk and
       provide the input file on checkmk's standard input instead, it is  gen-
       erally  recommended  to	provide	 it  as	an argument. Doing this	allows
       checkmk to be aware of the file's name, to place	references  to	it  in
       the  initial  comments  of  the C-language output, and to intersperse C
       #line directives	throughout, to facilitate in debugging problems	by di-
       recting the user	to the original	input file.

OPTIONS
       The  only officially supported option is	specifying a true value	(using
       Awk's definition	for "true") for	the variable clean_mode.  This	causes
       checkmk	not  to	place appropriate #line	directives in the source code,
       which some might	find to	be unnecessary clutter.

       The author recommends against the use of	this option, as	it will	 cause
       C  compilers and	debugging tools	to refer to lines in the automatically
       generated output, rather	than the original input	files to checkmk. This
       would  encourage	users to edit the output files instead of the original
       input files, would make it difficult for	intelligent editors or IDEs to
       pull  up	 the  right  file to edit, and could result in the fixes being
       overwritten when	the output files are regenerated.

       #line directives	are automatically supressed when  the  input  file  is
       provided	on standard input instead of as	a command-line argument.

BASIC EXAMPLE
       In  its	most  basic form, an input file	can be simply a	prologue and a
       test function. Anything that appears before the first test function  is
       in  the prologue, and will be copied into the output verbatim. The test
       function	is begun by a line in the form:

       #test test_name

       Where test_name is the name of your test	function. This will be used to
       name a C	function, so it	must be	a valid	C identifier.

       Here is a small,	complete example:

       --------------------------------------------------
       /* A complete test example */

       #include	<stdio.h>

       #test the_test
	   int nc;
	   const char msg[] = "\n\n    Hello, world!\n";

	   nc =	printf("%s", msg);
	   ck_assert(nc	== (sizeof(msg)	- 1)); /* for terminating NUL. */
       --------------------------------------------------

       If  you place the above into a file named basic_complete.ts and process
       it using	the following command:

       $ checkmk basic_complete.ts > basic_complete.c

       basic_complete.c	will contain output similar to:

       --------------------------------------------------
       /*
	* DO NOT EDIT THIS FILE. Generated by checkmk.
	* Edit the original source file	"in" instead.
	*/

       #include	<check.h>

       /* A complete test example */

       #include	<stdio.h>

       START_TEST(the_test)
       {
	   int nc;
	   const char msg[] = "\n\n    Hello, world!\n";

	   nc =	printf("%s", msg);
	   ck_assert(nc	== (sizeof(msg)	- 1)); /* for terminating NUL. */
       }
       END_TEST

       int main(void)
       {
	   Suite *s1 = suite_create("Core");
	   TCase *tc1_1	= tcase_create("Core");
	   SRunner *sr = srunner_create(s1);
	   int nf;

	   suite_add_tcase(s1, tc1_1);
	   tcase_add_test(tc1_1, the_test);

	   srunner_run_all(sr, CK_ENV);
	   nf =	srunner_ntests_failed(sr);
	   srunner_free(sr);

	   return nf ==	0 ? 0 :	1;
       }
       --------------------------------------------------

       In real usage, basic_complete.c would also contain #line	directives.

DIRECTIVE SUMMARY
       Here is a complete summary of all the  C-preprocessor-style  directives
       that are	understood by checkmk. See below for more details.

       # test test_name
       # test-signal(signal) test_name
       # test-exit(exit_code) test_name
       # test-loop(start, end) test_name
       # test-loop-signal(signal, start, end) test_name
       # test-loop-exit(exit_code, start, end) test_name
       # suite TestSuiteName
       # tcase TestCaseName
       # main-pre
       # main-post

       All  directives	are case-insensitive. Whitespace may appear at the be-
       ginning of the line before the #, between the # and the directive,  be-
       tween the directive and any argument, and at the	end of the line.

TEST-DEFINING DIRECTIVES
       Here  is	a more detailed	explanation of the directives that may be used
       to define test functions	and their containers.

   TEST	FUNCTIONS
       # test test_name
       # test-signal(signal) test_name
       # test-exit(exit_code) test_name
       # test-loop(start, end) test_name
       # test-loop-signal(signal, start, end) test_name
       # test-loop-exit(exit_code, start, end) test_name

       These are the most basic	directives for creating	a template  for	 input
       to  checkmk. They are the only directives that are required: there must
       be at least one #test* directive	appearing in the template, or  checkmk
       will fail with an error message.	The #test* directives may be specified
       several times, each one beginning the definition	of a  new  test	 func-
       tion.

       The  test_name  argument	will be	used as	the name of a test function in
       the C-language output, so it must be a valid C identifier. That is,  it
       must begin with an alphabetic character or the underscore (_), followed
       by optional alpha-numeric characters and/or underscores.

       Universal Character Names (introduced in	C99) are also allowed, of  the
       form \uXXXX or \UXXXXXXXX, where	the X's	represent hexadecimal digits.

       It  is  an  error to specify the	same test_name in more than one	#test*
       directive, regardless of	whether	they  are  associated  with  different
       test cases or suites.

       See  CHECKMK  IDENTIFIERS  for  the list	of identifiers which should be
       avoided for use as test function	names.

   TEST	SUITES
       # suite TestSuiteName

       This directive specifies	the name of the	test suite  (Suite  object  in
       the  Check  test	 framework)  to	which all future test cases (and their
       test functions) will be added.

       The TestSuiteName is a text string, and may contain any sort of charac-
       ters  at	 all  (other  than ASCII NUL character,	and the	newline, which
       would terminate the directive). Any leading or trailing whitespace will
       be omitted from the test	suite name.

       Starting	 a  new	 test suite also begins	a new test case, whose name is
       identical to the	new test suite.	This test case name may	be  overridden
       by a subsequent #tcase directive.

       Note  that a Suite object won't actually	be defined by checkmk in the C
       output, unless it is followed at	some point by a	#test directive	(with-
       out  an intervening #suite). It is not an error for a #suite to have no
       associated #test's; the #suite (and  any	 associated  #tcase's)	simply
       won't  result in	any action on the part of checkmk (and would therefore
       be useless).

       It is an	error for a #suite directive to	specify	the same (case	sensi-
       tive) suite multiple times, unless the previous uses were not instanti-
       ated by the presence of at least	one associated #test directive.

       If you do not specify a #suite directive	before the first #test	direc-
       tive,  checkmk performs the equivalent of an implicit #suite directive,
       with the	string "Core" as the value for TestSuiteName  (this  also  im-
       plies  a	 "Core"	test case object). This	is demonstrated	above in BASIC
       EXAMPLE.

   TEST	CASES
       # tcase TestCaseName

       This directive specifies	the name of the	test case (TCase object	in the
       Check test framework) to	which all future test functions	will be	added.

       The #tcase works	very in	a way very similar to #suite. The TestCaseName
       is a text string, and may contain arbitrary characters; and a TCase ob-
       ject  won't  actually be	defined	unless it is followed by an associated
       #test directive.

       It is an	error for a #tcase directive to	specify	the same (case	sensi-
       tive)  test  case multiple times, unless	the previous uses were not in-
       stantiated by the presence of at	least one associated #test directive.

       See also	the #suite directive, described	above.

USER CODE IN MAIN()
       The C main() is automatically generated by checkmk, defining the	neces-
       sary  SRunner's,	Suite's, and TCase's required by the test-defining di-
       rectives	specified by the user.

       For most	situations, this completely automated main() is	quite suitable
       as-is. However, there are situations where one might wish to add	custom
       code to the main(). For instance, if the	user wishes to:

       o change	the test timeout value via tcase_set_timeout(),

       o specify Check's "no-fork-mode"	via srunner_set_fork_status(),

       o set up	test fixtures for some test cases, via	tcase_add_checked_fix-
	 ture()	or tcase_add_unchecked_fixture(),

       o set  up  test	logging	 for  the  suite runner, via srunner_set_log()
	 or srunner_set_xml(), or

       o perform custom	wrap-up	after the test suites have been	run.

       For these purposes, the #main-pre and #main-post	directives  have  been
       provided.

   MAIN() PROLOGUE
       # main-pre

       The text	following this directive will be placed	verbatim into the body
       of the generated	main() function, just after checkmk's own local	 vari-
       able declarations, and before any test running has taken	place (indeed,
       before even the relationships between the tests,	test cases,  and  test
       suites  have  been set up, though that fact shouldn't make much differ-
       ence). Since checkmk has	only just finished making its declarations, it
       is permissible, even under strict 1990 ISO C guidelines,	to make	custom
       variable	declarations here.

       Unlike the previously-described directives, #main-pre may be  specified
       at  most	 once. It may not be preceded by the #main-post	directive, and
       no #suite, #tcase, or #test directive may appear	after it.

       #main-pre is a good place to tweak settings or set up test fixtures. Of
       course, in order	to do so, you need to know what	names checkmk has used
       to instantiate the SRunner's, Suite's, and TCase's.

   CHECKMK IDENTIFIERS
       Pointers	to Suite's are declared	using the pattern sX,  where  X	 is  a
       number  that starts at 1, and is	incremented for	each subsequent	#suite
       directive.  s1 always exists, and contains the test  function  declared
       by  the	first #test directive. If that directive was not preceded by a
       #suite, it will be given	the name "Core".

       Pointers	to TCase's are declared	using the pattern tcX_Y, where X  cor-
       responds	to the number used for the name	of the Suite that will contain
       this TCase; and Y is a number that starts at 1 for each new Suite,  and
       is incremented for each TCase in	that Suite.

       A pointer to SRunner is declared	using the identifier sr; there is also
       an integer named	nf which holds the number of test failures (after  the
       tests have run).

       For obvious reasons, the	user should not	attempt	to declare local iden-
       tifiers in main(), or define any	macros or test functions, whose	 names
       might conflict with the local variable names used by checkmk. To	summa-
       rize, these names are:

       sX

       tcX_Y

       sr

       nf.

   MAIN() EPILOGUE
       # main-post

       Though it is not	as useful, checkmk also	provides a  #main-post	direc-
       tive  to	 insert	custom code at the end of main(), after	the tests have
       run. This could be used to clean	up resources that  were	 allocated  in
       the  prologue,  or  to  print information about the failed tests, or to
       provide a custom	exit status code.

       Note that, if you make use of this directive, checkmk will not  provide
       a return	statement: you will need to provide one	yourself.

       The  #main-post	directive  may not be followed by any other directives
       recognized by checkmk.

COMPREHENSIVE EXAMPLE
       Now that	you've gotten the detailed descriptions	of the various	direc-
       tives,  let's  see  it all put to action	with this fairly comprehensive
       template.

       --------------------------------------------------
       #include	"mempool.h"  /*	defines	MEMPOOLSZ, prototypes for
				mempool_init() and mempool_free() */

       void *mempool;

       void mp_setup(void)
       {
	   mempool = mempool_init(MEMPOOLSZ);
	   ck_assert_msg(mempool != NULL, "Couldn't allocate mempool.");
       }

       void mp_teardown(void)
       {
	   mempool_free(mempool);
       }

       /* end of prologue */

       #suite Mempool

       #tcase MP Init

       #test mempool_init_zero_test
	   mempool = mempool_init(0);
	   ck_assert_msg(mempool == NULL, "Allocated a zero-sized mempool!");
	   ck_assert_msg(mempool_error(), "Didn't get an error for zero	alloc.");

       /* "MP Util" TCase uses checked fixture.	*/
       #tcase MP Util

       #test mempool_copy_test
	   void	*cp = mempool_copy(mempool);
	   ck_assert_msg(cp != NULL, "Couldn't perform mempool copy.");
	   ck_assert_msg(cp != mempool,	"Copy returned original	pointer!");

       #test mempool_size_test
	   ck_assert(mempool_getsize(mempool) == MEMPOOLSZ);

       #main-pre
	   tcase_add_checked_fixture(tc1_2, mp_setup, mp_teardown);
	   srunner_set_log(sr, "mplog.txt");

       #main-post
	   if (nf != 0)	{
	     printf("Hey, something's wrong! %d	whole tests failed!\n",	nf);
	   }
	   return 0; /*	Harness	checks for output, always return success
			regardless. */
       --------------------------------------------------

       Plugging	this into checkmk, we'll get output roughly like  the  follow-
       ing:

       --------------------------------------------------
       /*
	* DO NOT EDIT THIS FILE. Generated by checkmk.
	* Edit the original source file	"comprehensive.ts" instead.
	*/

       #include	<check.h>

       #include	"mempool.h"

       void *mempool;

       void mp_setup(void)
       {
       ...
       }

       void mp_teardown(void)
       {
       ...
       }

       /* end of prologue */

       START_TEST(mempool_init_zero_test)
       {
       ...
       }
       END_TEST

       START_TEST(mempool_copy_test)
       {
       ...
       }
       END_TEST

       START_TEST(mempool_size_test)
       {
       ...
       }
       END_TEST

       int main(void)
       {
	   Suite *s1 = suite_create("Mempool");
	   TCase *tc1_1	= tcase_create("MP Init");
	   TCase *tc1_2	= tcase_create("MP Util");
	   SRunner *sr = srunner_create(s1);
	   int nf;

	   /* User-specified pre-run code */
	   tcase_add_checked_fixture(tc1_2, mp_setup, mp_teardown);
	   srunner_set_log(sr, "mplog.txt");

	   suite_add_tcase(s1, tc1_1);
	   tcase_add_test(tc1_1, mempool_init_zero_test);
	   suite_add_tcase(s1, tc1_2);
	   tcase_add_test(tc1_2, mempool_copy_test);
	   tcase_add_test(tc1_2, mempool_size_test);

	   srunner_run_all(sr, CK_ENV);
	   nf =	srunner_ntests_failed(sr);
	   srunner_free(sr);

	   /* User-specified post-run code */
	   if (nf != 0)	{
	     printf("Hey, something's wrong! %d	whole tests failed!\n",	nf);
	   }
	   return 0; /*	Harness	checks for output, always return success
			regardless. */
       }
       --------------------------------------------------

AUTHOR
       checkmk and this	manual were written by Micah J Cowan.

       Copyright (C) 2006, 2010	Micah J	Cowan.

			       09 February 2010			    CHECKMK(1)

NAME | SYNOPSIS | DESCRIPTION | OPTIONS | BASIC EXAMPLE | DIRECTIVE SUMMARY | TEST-DEFINING DIRECTIVES | USER CODE IN MAIN() | COMPREHENSIVE EXAMPLE | AUTHOR

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=checkmk&sektion=1&manpath=FreeBSD+12.1-RELEASE+and+Ports>

home | help