Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
CHECKMK(1)							    CHECKMK(1)

NAME
       checkmk	-  Awk	script	for  generating	 C unit	tests for use with the
       Check unit testing framework.

SYNOPSIS
       checkmk [ clean_mode=1 ]	[ input-file ]

DESCRIPTION
       Generate	C-language source files	containing unit	tests for use with the
       Check  unit  testing  framework.	 The aim of this script	is to automate
       away some of the	typical	boilerplate one	must write when	writing	a test
       suite  using  Check:  specifically,  the	 instantiation	of an SRunner,
       Suite(s), and TCase(s), and the building	of relationships between these
       objects and the test functions.

       This  tool  is  intended	 to be used by those who are familiar with the
       Check unit testing framework. Familiarity with the  framework  will  be
       assumed throughout this manual.

       The  Check framework, along with	information regarding it, is available
       at				     https://libcheck.github.io/check/
       <URL:https://libcheck.github.io/check/>.

       The  input-file	argument to checkmk uses a simple, C-preprocessor-like
       syntax to declare test functions, and to	describe  their	 relationships
       to  Suites  and TCases in Check.	 checkmk then uses this	information to
       automatically write a main() function containing	all of	the  necessary
       declarations,  and  whatever code is needed to run the test suites. The
       final C-language	output is printed to checkmk's standard	output.

       Facilities are provided for the insertion of user code into the	gener-
       ated  main() function, to provide for the use of	logging, test fixtures
       or specialized exit values.

       While it	is possible to omit the	input-file  argument  to  checkmk  and
       provide	the input file on checkmk's standard input instead, it is gen-
       erally recommended to provide it	as  an	argument.  Doing  this	allows
       checkmk	to  be	aware of the file's name, to place references to it in
       the initial comments of the C-language output,  and  to	intersperse  C
       #line directives	throughout, to facilitate in debugging problems	by di-
       recting the user	to the original	input file.

OPTIONS
       The only	officially supported option is specifying a true value	(using
       Awk's  definition  for "true") for the variable clean_mode. This	causes
       checkmk not to place appropriate	#line directives in the	 source	 code,
       which some might	find to	be unnecessary clutter.

       The  author recommends against the use of this option, as it will cause
       C compilers and debugging tools to refer	to lines in the	 automatically
       generated output, rather	than the original input	files to checkmk. This
       would encourage users to	edit the output	files instead of the  original
       input files, would make it difficult for	intelligent editors or IDEs to
       pull up the right file to edit, and could result	 in  the  fixes	 being
       overwritten when	the output files are regenerated.

       #line  directives  are  automatically  supressed	when the input file is
       provided	on standard input instead of as	a command-line argument.

BASIC EXAMPLE
       In its most basic form, an input	file can be simply a  prologue	and  a
       test  function. Anything	that appears before the	first test function is
       in the prologue,	and will be copied into	the output verbatim. The  test
       function	is begun by a line in the form:

       #test test_name

       Where test_name is the name of your test	function. This will be used to
       name a C	function, so it	must be	a valid	C identifier.

       Here is a small,	complete example:

       --------------------------------------------------
       /* A complete test example */

       #include	<stdio.h>

       #test the_test
	   int nc;
	   const char msg[] = "\n\n    Hello, world!\n";

	   nc =	printf("%s", msg);
	   ck_assert(nc	== (sizeof(msg)	- 1)); /* for terminating NUL. */
       --------------------------------------------------

       If you place the	above into a file named	basic_complete.ts and  process
       it using	the following command:

       $ checkmk basic_complete.ts > basic_complete.c

       basic_complete.c	will contain output similar to:

       --------------------------------------------------
       /*
	* DO NOT EDIT THIS FILE. Generated by checkmk.
	* Edit the original source file	"in" instead.
	*/

       #include	<check.h>

       /* A complete test example */

       #include	<stdio.h>

       START_TEST(the_test)
       {
	   int nc;
	   const char msg[] = "\n\n    Hello, world!\n";

	   nc =	printf("%s", msg);
	   ck_assert(nc	== (sizeof(msg)	- 1)); /* for terminating NUL. */
       }
       END_TEST

       int main(void)
       {
	   Suite *s1 = suite_create("Core");
	   TCase *tc1_1	= tcase_create("Core");
	   SRunner *sr = srunner_create(s1);
	   int nf;

	   suite_add_tcase(s1, tc1_1);
	   tcase_add_test(tc1_1, the_test);

	   srunner_run_all(sr, CK_ENV);
	   nf =	srunner_ntests_failed(sr);
	   srunner_free(sr);

	   return nf ==	0 ? 0 :	1;
       }
       --------------------------------------------------

       In real usage, basic_complete.c would also contain #line	directives.

DIRECTIVE SUMMARY
       Here  is	 a complete summary of all the C-preprocessor-style directives
       that are	understood by checkmk. See below for more details.

       # test test_name
       # test-signal(signal) test_name
       # test-exit(exit_code) test_name
       # test-loop(start, end) test_name
       # test-loop-signal(signal, start, end) test_name
       # test-loop-exit(exit_code, start, end) test_name
       # suite TestSuiteName
       # tcase TestCaseName
       # main-pre
       # main-post

       All directives are case-insensitive. Whitespace may appear at  the  be-
       ginning	of the line before the #, between the #	and the	directive, be-
       tween the directive and any argument, and at the	end of the line.

TEST-DEFINING DIRECTIVES
       Here is a more detailed explanation of the directives that may be  used
       to define test functions	and their containers.

   TEST	FUNCTIONS
       # test test_name
       # test-signal(signal) test_name
       # test-exit(exit_code) test_name
       # test-loop(start, end) test_name
       # test-loop-signal(signal, start, end) test_name
       # test-loop-exit(exit_code, start, end) test_name

       These  are  the most basic directives for creating a template for input
       to checkmk. They	are the	only directives	that are required: there  must
       be  at least one	#test* directive appearing in the template, or checkmk
       will fail with an error message.	The #test* directives may be specified
       several	times,	each  one beginning the	definition of a	new test func-
       tion.

       The test_name argument will be used as the name of a test  function  in
       the  C-language output, so it must be a valid C identifier. That	is, it
       must begin with an alphabetic character or the underscore (_), followed
       by optional alpha-numeric characters and/or underscores.

       Universal  Character Names (introduced in C99) are also allowed,	of the
       form \uXXXX or \UXXXXXXXX, where	the X's	represent hexadecimal digits.

       It is an	error to specify the same test_name in more  than  one	#test*
       directive,  regardless  of  whether  they are associated	with different
       test cases or suites.

       See CHECKMK IDENTIFIERS for the list of	identifiers  which  should  be
       avoided for use as test function	names.

   TEST	SUITES
       # suite TestSuiteName

       This  directive	specifies  the name of the test	suite (Suite object in
       the Check test framework) to which all future  test  cases  (and	 their
       test functions) will be added.

       The TestSuiteName is a text string, and may contain any sort of charac-
       ters at all (other than ASCII NUL character,  and  the  newline,	 which
       would terminate the directive). Any leading or trailing whitespace will
       be omitted from the test	suite name.

       Starting	a new test suite also begins a new test	case,  whose  name  is
       identical  to the new test suite. This test case	name may be overridden
       by a subsequent #tcase directive.

       Note that a Suite object	won't actually be defined by checkmk in	the  C
       output, unless it is followed at	some point by a	#test directive	(with-
       out an intervening #suite). It is not an	error for a #suite to have  no
       associated  #test's;  the  #suite  (and any associated #tcase's)	simply
       won't result in any action on the part of checkmk (and would  therefore
       be useless).

       It  is an error for a #suite directive to specify the same (case	sensi-
       tive) suite multiple times, unless the previous uses were not instanti-
       ated by the presence of at least	one associated #test directive.

       If  you do not specify a	#suite directive before	the first #test	direc-
       tive, checkmk performs the equivalent of	an implicit #suite  directive,
       with  the  string  "Core" as the	value for TestSuiteName	(this also im-
       plies a "Core" test case	object). This is demonstrated above  in	 BASIC
       EXAMPLE.

   TEST	CASES
       # tcase TestCaseName

       This directive specifies	the name of the	test case (TCase object	in the
       Check test framework) to	which all future test functions	will be	added.

       The #tcase works	very in	a way very similar to #suite. The TestCaseName
       is a text string, and may contain arbitrary characters; and a TCase ob-
       ject won't actually be defined unless it	is followed by	an  associated
       #test directive.

       It  is an error for a #tcase directive to specify the same (case	sensi-
       tive) test case multiple	times, unless the previous uses	were  not  in-
       stantiated by the presence of at	least one associated #test directive.

       See also	the #suite directive, described	above.

USER CODE IN MAIN()
       The C main() is automatically generated by checkmk, defining the	neces-
       sary SRunner's, Suite's,	and TCase's required by	the test-defining  di-
       rectives	specified by the user.

       For most	situations, this completely automated main() is	quite suitable
       as-is. However, there are situations where one might wish to add	custom
       code to the main(). For instance, if the	user wishes to:

       o change	the test timeout value via tcase_set_timeout(),

       o specify Check's "no-fork-mode"	via srunner_set_fork_status(),

       o set  up test fixtures for some	test cases, via	tcase_add_checked_fix-
	 ture()	or tcase_add_unchecked_fixture(),

       o set up	test logging  for  the	suite  runner,	via  srunner_set_log()
	 or srunner_set_xml(), or

       o perform custom	wrap-up	after the test suites have been	run.

       For  these  purposes, the #main-pre and #main-post directives have been
       provided.

   MAIN() PROLOGUE
       # main-pre

       The text	following this directive will be placed	verbatim into the body
       of  the generated main()	function, just after checkmk's own local vari-
       able declarations, and before any test running has taken	place (indeed,
       before  even  the relationships between the tests, test cases, and test
       suites have been	set up,	though that fact shouldn't make	 much  differ-
       ence). Since checkmk has	only just finished making its declarations, it
       is permissible, even under strict 1990 ISO C guidelines,	to make	custom
       variable	declarations here.

       Unlike  the previously-described	directives, #main-pre may be specified
       at most once. It	may not	be preceded by the #main-post  directive,  and
       no #suite, #tcase, or #test directive may appear	after it.

       #main-pre is a good place to tweak settings or set up test fixtures. Of
       course, in order	to do so, you need to know what	names checkmk has used
       to instantiate the SRunner's, Suite's, and TCase's.

   CHECKMK IDENTIFIERS
       Pointers	 to  Suite's  are  declared using the pattern sX, where	X is a
       number that starts at 1,	and is incremented for each subsequent	#suite
       directive.   s1	always exists, and contains the	test function declared
       by the first #test directive. If	that directive was not preceded	 by  a
       #suite, it will be given	the name "Core".

       Pointers	 to TCase's are	declared using the pattern tcX_Y, where	X cor-
       responds	to the number used for the name	of the Suite that will contain
       this  TCase; and	Y is a number that starts at 1 for each	new Suite, and
       is incremented for each TCase in	that Suite.

       A pointer to SRunner is declared	using the identifier sr; there is also
       an  integer named nf which holds	the number of test failures (after the
       tests have run).

       For obvious reasons, the	user should not	attempt	to declare local iden-
       tifiers	in main(), or define any macros	or test	functions, whose names
       might conflict with the local variable names used by checkmk. To	summa-
       rize, these names are:

       sX

       tcX_Y

       sr

       nf.

   MAIN() EPILOGUE
       # main-post

       Though  it  is not as useful, checkmk also provides a #main-post	direc-
       tive to insert custom code at the end of	main(),	after the  tests  have
       run.  This  could  be used to clean up resources	that were allocated in
       the prologue, or	to print information about the	failed	tests,	or  to
       provide a custom	exit status code.

       Note  that, if you make use of this directive, checkmk will not provide
       a return	statement: you will need to provide one	yourself.

       The #main-post directive	may not	be followed by	any  other  directives
       recognized by checkmk.

COMPREHENSIVE EXAMPLE
       Now  that you've	gotten the detailed descriptions of the	various	direc-
       tives, let's see	it all put to action with  this	 fairly	 comprehensive
       template.

       --------------------------------------------------
       #include	"mempool.h"  /*	defines	MEMPOOLSZ, prototypes for
				mempool_init() and mempool_free() */

       void *mempool;

       void mp_setup(void)
       {
	   mempool = mempool_init(MEMPOOLSZ);
	   ck_assert_msg(mempool != NULL, "Couldn't allocate mempool.");
       }

       void mp_teardown(void)
       {
	   mempool_free(mempool);
       }

       /* end of prologue */

       #suite Mempool

       #tcase MP Init

       #test mempool_init_zero_test
	   mempool = mempool_init(0);
	   ck_assert_msg(mempool == NULL, "Allocated a zero-sized mempool!");
	   ck_assert_msg(mempool_error(), "Didn't get an error for zero	alloc.");

       /* "MP Util" TCase uses checked fixture.	*/
       #tcase MP Util

       #test mempool_copy_test
	   void	*cp = mempool_copy(mempool);
	   ck_assert_msg(cp != NULL, "Couldn't perform mempool copy.");
	   ck_assert_msg(cp != mempool,	"Copy returned original	pointer!");

       #test mempool_size_test
	   ck_assert(mempool_getsize(mempool) == MEMPOOLSZ);

       #main-pre
	   tcase_add_checked_fixture(tc1_2, mp_setup, mp_teardown);
	   srunner_set_log(sr, "mplog.txt");

       #main-post
	   if (nf != 0)	{
	     printf("Hey, something's wrong! %d	whole tests failed!\n",	nf);
	   }
	   return 0; /*	Harness	checks for output, always return success
			regardless. */
       --------------------------------------------------

       Plugging	 this  into checkmk, we'll get output roughly like the follow-
       ing:

       --------------------------------------------------
       /*
	* DO NOT EDIT THIS FILE. Generated by checkmk.
	* Edit the original source file	"comprehensive.ts" instead.
	*/

       #include	<check.h>

       #include	"mempool.h"

       void *mempool;

       void mp_setup(void)
       {
       ...
       }

       void mp_teardown(void)
       {
       ...
       }

       /* end of prologue */

       START_TEST(mempool_init_zero_test)
       {
       ...
       }
       END_TEST

       START_TEST(mempool_copy_test)
       {
       ...
       }
       END_TEST

       START_TEST(mempool_size_test)
       {
       ...
       }
       END_TEST

       int main(void)
       {
	   Suite *s1 = suite_create("Mempool");
	   TCase *tc1_1	= tcase_create("MP Init");
	   TCase *tc1_2	= tcase_create("MP Util");
	   SRunner *sr = srunner_create(s1);
	   int nf;

	   /* User-specified pre-run code */
	   tcase_add_checked_fixture(tc1_2, mp_setup, mp_teardown);
	   srunner_set_log(sr, "mplog.txt");

	   suite_add_tcase(s1, tc1_1);
	   tcase_add_test(tc1_1, mempool_init_zero_test);
	   suite_add_tcase(s1, tc1_2);
	   tcase_add_test(tc1_2, mempool_copy_test);
	   tcase_add_test(tc1_2, mempool_size_test);

	   srunner_run_all(sr, CK_ENV);
	   nf =	srunner_ntests_failed(sr);
	   srunner_free(sr);

	   /* User-specified post-run code */
	   if (nf != 0)	{
	     printf("Hey, something's wrong! %d	whole tests failed!\n",	nf);
	   }
	   return 0; /*	Harness	checks for output, always return success
			regardless. */
       }
       --------------------------------------------------

AUTHOR
       checkmk and this	manual were written by Micah J Cowan.

       Copyright (C) 2006, 2010	Micah J	Cowan.

			       09 February 2010			    CHECKMK(1)

NAME | SYNOPSIS | DESCRIPTION | OPTIONS | BASIC EXAMPLE | DIRECTIVE SUMMARY | TEST-DEFINING DIRECTIVES | USER CODE IN MAIN() | COMPREHENSIVE EXAMPLE | AUTHOR

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=checkmk&manpath=FreeBSD+12.1-RELEASE+and+Ports>

home | help