Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
common_test(3)		   Erlang Module Definition		common_test(3)

NAME
       common_test - A framework for automated testing of any target nodes.

DESCRIPTION
       The  Common  Test framework is an environment for implementing and per-
       forming automatic and semi-automatic execution of test cases.

       In brief, Common	Test supports:

	 * Automated execution of test suites (sets of test cases)

	 * Logging of events during execution

	 * HTML	presentation of	test suite results

	 * HTML	presentation of	test suite code

	 * Support functions for test suite authors

	 * Step-by-step	execution of test cases

       The following section describes the mandatory and optional  test	 suite
       functions  that	Common	Test calls during test execution. For more de-
       tails, see section Writing Test Suites in the User's Guide.

TEST CASE CALLBACK FUNCTIONS
       The following functions define the callback interface for a test	suite.

EXPORTS
       Module:all() -> Tests | {skip,Reason}

	      Types:

		 Tests	=  [TestCase  |	 {group,GroupName}   |	 {group,Group-
		 Name,Properties} | {group,GroupName,Properties,SubGroups}]
		 TestCase = atom()
		 GroupName = atom()
		 Properties = [parallel	| sequence | Shuffle | {RepeatType,N}]
		 | default
		 SubGroups  =  [{GroupName,Properties}	|   {GroupName,Proper-
		 ties,SubGroups}]
		 Shuffle = shuffle | {shuffle,Seed}
		 Seed =	{integer(),integer(),integer()}
		 RepeatType   =	 repeat	 |  repeat_until_all_ok	 |  repeat_un-
		 til_all_fail |	repeat_until_any_ok | repeat_until_any_fail
		 N = integer() | forever
		 Reason	= term()

	      MANDATORY

	      Returns the list of all test cases and test case groups  in  the
	      test  suite  module to be	executed. This list also specifies the
	      order the	cases and groups are executed by Common	Test.  A  test
	      case  is represented by an atom, the name	of the test case func-
	      tion. A test case	group is represented by	a group	 tuple,	 where
	      GroupName,  an  atom,  is	 the  name  of	the  group (defined in
	      groups/0). Execution properties for groups can  also  be	speci-
	      fied,  both  for a top-level group and for any of	its subgroups.
	      Group execution properties specified here	override properties in
	      the  group  definition  (see groups/0). (With value default, the
	      group definition properties are used).

	      If {skip,Reason} is returned, all	test cases in the  module  are
	      skipped and Reason is printed on the HTML	result page.

	      For  details  on	groups,	 see  section  Test Case Groups	in the
	      User's Guide.

       Module:groups() -> GroupDefs

	      Types:

		 GroupDefs = [Group]
		 Group = {GroupName,Properties,GroupsAndTestCases}
		 GroupName = atom()
		 Properties = [parallel	| sequence | Shuffle | {RepeatType,N}]
		 GroupsAndTestCases = [Group | {group,GroupName} | TestCase]
		 TestCase = atom()
		 Shuffle = shuffle | {shuffle,Seed}
		 Seed =	{integer(),integer(),integer()}
		 RepeatType  =	repeat	|  repeat_until_all_ok	|   repeat_un-
		 til_all_fail |	repeat_until_any_ok | repeat_until_any_fail
		 N = integer() | forever

	      OPTIONAL

	      Defines  test  case  groups.  For	details, see section Test Case
	      Groups in	the User's Guide.

       Module:suite() -> [Info]

	      Types:

		 Info  =   {timetrap,Time}   |	 {require,Required}   |	  {re-
		 quire,Name,Required}  | {userdata,UserData} | {silent_connec-
		 tions,Conns} |	{stylesheet,CSSFile} | {ct_hooks, CTHs}
		 Time =	TimeVal	| TimeFunc
		 TimeVal = MilliSec  |	{seconds,integer()}  |	{minutes,inte-
		 ger()}	| {hours,integer()}
		 TimeFunc = {Mod,Func,Args} | Fun
		 MilliSec = integer()
		 Mod = atom()
		 Func =	atom()
		 Args =	list()
		 Fun = fun()
		 Required  =  Key  |  {Key,SubKeys} | {Key,SubKey} | {Key,Sub-
		 Key,SubKeys}
		 Key = atom()
		 SubKeys = SubKey | [SubKey]
		 SubKey	= atom()
		 Name =	atom()
		 UserData = term()
		 Conns = [atom()]
		 CSSFile = string()
		 CTHs =	[CTHModule |
		  {CTHModule, CTHInitArgs} |
		  {CTHModule, CTHInitArgs, CTHPriority}]
		 CTHModule = atom()
		 CTHInitArgs = term()

	      OPTIONAL

	      The test suite information function. Returns a  list  of	tagged
	      tuples specifying	various	properties related to the execution of
	      this test	suite (common for all test cases in the	suite).

	      Tag timetrap sets	the maximum time that each test	 case  is  al-
	      lowed    to    execute	(including   init_per_testcase/2   and
	      end_per_testcase/2). If the timetrap time	is exceeded, the  test
	      case fails with reason timetrap_timeout. A TimeFunc function can
	      be used to set a new timetrap by returning  a  TimeVal.  It  can
	      also  be	used to	trigger	a timetrap time-out by,	at some	point,
	      returning	a value	other than a TimeVal. For details, see section
	      Timetrap Time-Outs in the	User's Guide.

	      Tag  require  specifies configuration variables required by test
	      cases (or	configuration functions) in the	suite. If the required
	      configuration  variables	are not	found in any of	the configura-
	      tion files, all test cases are skipped. For  details  about  the
	      require functionality, see funtion ct:require/1,2.

	      With  userdata,  the user	can specify any	test suite-related in-
	      formation, which can be read by calling ct:userdata/2.

	      Tag ct_hooks specifies the Common	Test Hooks to be run with this
	      suite.

	      Other tuples than	the ones defined are ignored.

	      For  details about the test suite	information function, see sec-
	      tion Test	Suite Information Function in the User's Guide.

       Module:init_per_suite(Config)   ->   NewConfig	|   {skip,Reason}    |
       {skip_and_save,Reason,SaveConfig}

	      Types:

		 Config	= NewConfig = SaveConfig = [{Key,Value}]
		 Key = atom()
		 Value = term()
		 Reason	= term()

	      OPTIONAL;	if this	function is defined, then end_per_suite/1 must
	      also be defined.

	      This configuration function is called as the first  function  in
	      the suite. It typically contains initializations that are	common
	      for all test cases in the	suite, and  that  must	only  be  done
	      once.  Parameter	Config	is  the	configuration data that	can be
	      modified.	Whatever is returned from this function	 is  specified
	      as  Config  to all configuration functions and test cases	in the
	      suite.

	      If {skip,Reason} is returned, all	test cases in  the  suite  are
	      skipped and Reason is printed in the overview log	for the	suite.

	      For  information	on  save_config	and skip_and_save, see section
	      Saving Configuration Data	in the User's Guide.

       Module:end_per_suite(Config) -> term() |	{save_config,SaveConfig}

	      Types:

		 Config	= SaveConfig = [{Key,Value}]
		 Key = atom()
		 Value = term()

	      OPTIONAL;	if this	function  is  defined,	then  init_per_suite/1
	      must also	be defined.

	      This  function  is called	as the last test case in the suite. It
	      is meant to be used for cleaning up after	init_per_suite/1.

	      For information on save_config, see section Saving Configuration
	      Data in the User's Guide.

       Module:group(GroupName) -> [Info]

	      Types:

		 Info	=   {timetrap,Time}   |	  {require,Required}   |  {re-
		 quire,Name,Required} |	{userdata,UserData} |  {silent_connec-
		 tions,Conns} |	{stylesheet,CSSFile} | {ct_hooks, CTHs}
		 Time =	TimeVal	| TimeFunc
		 TimeVal  =  MilliSec  |  {seconds,integer()} |	{minutes,inte-
		 ger()}	| {hours,integer()}
		 TimeFunc = {Mod,Func,Args} | Fun
		 MilliSec = integer()
		 Mod = atom()
		 Func =	atom()
		 Args =	list()
		 Fun = fun()
		 Required = Key	| {Key,SubKeys}	 |  {Key,Subkey}  |  {Key,Sub-
		 key,SubKeys}
		 Key = atom()
		 SubKeys = SubKey | [SubKey]
		 SubKey	= atom()
		 Name =	atom()
		 UserData = term()
		 Conns = [atom()]
		 CSSFile = string()
		 CTHs =	[CTHModule |
		  {CTHModule, CTHInitArgs} |
		  {CTHModule, CTHInitArgs, CTHPriority}]
		 CTHModule = atom()
		 CTHInitArgs = term()

	      OPTIONAL

	      The  test	case group information function. It is supposed	to re-
	      turn a list of tagged tuples that	specify	various	properties re-
	      lated  to	 the execution of a test case group (that is, its test
	      cases and	subgroups). Properties set by group/1 override proper-
	      ties with	the same key that have been set	previously by suite/0.

	      Tag  timetrap  sets  the maximum time that each test case	is al-
	      lowed   to   execute    (including    init_per_testcase/2	   and
	      end_per_testcase/2).  If the timetrap time is exceeded, the test
	      case fails with reason timetrap_timeout. A TimeFunc function can
	      be  used	to  set	 a new timetrap	by returning a TimeVal.	It can
	      also be used to trigger a	timetrap time-out by, at  some	point,
	      returning	a value	other than a TimeVal. For details, see section
	      Timetrap Time-Outs in the	User's Guide.

	      Tag require specifies configuration variables required  by  test
	      cases (or	configuration functions) in the	suite. If the required
	      configuration variables are not found in any of  the  configura-
	      tion  files,  all	 test cases in this group are skipped. For de-
	      tails about  the	require	 functionality,	 see  function	ct:re-
	      quire/1,2.

	      With  userdata, the user can specify any test case group related
	      information that can be read by calling ct:userdata/2.

	      Tag ct_hooks specifies the Common	Test Hooks to be run with this
	      suite.

	      Other tuples than	the ones defined are ignored.

	      For  details about the test case group information function, see
	      section Group Information	Function in the	User's Guide.

       Module:init_per_group(GroupName,	Config)	-> NewConfig | {skip,Reason}

	      Types:

		 GroupName = atom()
		 Config	= NewConfig = [{Key,Value}]
		 Key = atom()
		 Value = term()
		 Reason	= term()

	      OPTIONAL;	if this	function is defined, then end_per_group/2 must
	      also be defined.

	      This configuration function is called before execution of	a test
	      case group. It typically contains	initializations	that are  com-
	      mon for all test cases and subgroups in the group, and that must
	      only be performed	once. GroupName	is the name of the  group,  as
	      specified	in the group definition	(see groups/0).	Parameter Con-
	      fig is the configuration data that can be	modified.  The	return
	      value  of	this function is given as Config to all	test cases and
	      subgroups	in the group.

	      If {skip,Reason} is returned, all	test cases in  the  group  are
	      skipped and Reason is printed in the overview log	for the	group.

	      For  information	about  test case groups, see section Test Case
	      Groups in	the User's Guide.

       Module:end_per_group(GroupName, Config) -> term()  |  {return_group_re-
       sult,Status}

	      Types:

		 GroupName = atom()
		 Config	= [{Key,Value}]
		 Key = atom()
		 Value = term()
		 Status	= ok | skipped | failed

	      OPTIONAL;	 if  this  function  is	defined, then init_per_group/2
	      must also	be defined.

	      This function is called after the	execution of a test case group
	      is  finished.  It	 is  meant  to	be  used for cleaning up after
	      init_per_group/2.	A status value for a nested  subgroup  can  be
	      returned	with  {return_group_result,Status}.  The status	can be
	      retrieved	in end_per_group/2 for the group on the	 level	above.
	      The status is also used by Common	Test for deciding if execution
	      of a group is to proceed if property sequence or	repeat_until_*
	      is set.

	      For details about	test case groups, see section Test Case	Groups
	      in the User's Guide.

       Module:init_per_testcase(TestCase, Config) -> NewConfig | {fail,Reason}
       | {skip,Reason}

	      Types:

		  TestCase = atom()
		  Config = NewConfig = [{Key,Value}]
		  Key =	atom()
		  Value	= term()
		  Reason = term()

	      OPTIONAL;	 if  this function is defined, then end_per_testcase/2
	      must also	be defined.

	      This function is called before each test case. Argument TestCase
	      is  the test case	name, and Config (list of key-value tuples) is
	      the configuration	data that can be modified. The NewConfig  list
	      returned from this function is given as Config to	the test case.
	      If {fail,Reason} is returned, the	test case is marked as	failed
	      without being executed.

	      If  {skip,Reason}	is returned, the test case is skipped and Rea-
	      son is printed in	the overview log for the suite.

       Module:end_per_testcase(TestCase, Config) -> term() |  {fail,Reason}  |
       {save_config,SaveConfig}

	      Types:

		 TestCase = atom()
		 Config	= SaveConfig = [{Key,Value}]
		 Key = atom()
		 Value = term()
		 Reason	= term()

	      OPTIONAL;	 if this function is defined, then init_per_testcase/2
	      must also	be defined.

	      This function is called after each test case, and	can be used to
	      clean up after init_per_testcase/2 and the test case. Any	return
	      value (besides {fail,Reason}  and	 {save_config,SaveConfig})  is
	      ignored.	By  returning  {fail,Reason},  TestCase	 is  marked as
	      faulty (even though it was successful in the sense that  it  re-
	      turned a value instead of	terminating).

	      For information on save_config, see section Saving Configuration
	      Data in the User's Guide.

       Module:Testcase() -> [Info]

	      Types:

		 Info  =   {timetrap,Time}   |	 {require,Required}   |	  {re-
		 quire,Name,Required}  | {userdata,UserData} | {silent_connec-
		 tions,Conns}
		 Time =	TimeVal	| TimeFunc
		 TimeVal = MilliSec  |	{seconds,integer()}  |	{minutes,inte-
		 ger()}	| {hours,integer()}
		 TimeFunc = {Mod,Func,Args} | Fun
		 MilliSec = integer()
		 Mod = atom()
		 Func =	atom()
		 Args =	list()
		 Fun = fun()
		 Required  =  Key  |  {Key,SubKeys} | {Key,Subkey} | {Key,Sub-
		 key,SubKeys}
		 Key = atom()
		 SubKeys = SubKey | [SubKey]
		 SubKey	= atom()
		 Name =	atom()
		 UserData = term()
		 Conns = [atom()]

	      OPTIONAL

	      The test case information	function. It is	supposed to  return  a
	      list of tagged tuples that specify various properties related to
	      the execution of this particular test case.  Properties  set  by
	      Testcase/0  override properties set previously for the test case
	      by group/1 or suite/0.

	      Tag timetrap sets	the maximum time that the test case is allowed
	      to  execute.  If	the  timetrap  time is exceeded, the test case
	      fails  with  reason  timetrap_timeout.  init_per_testcase/2  and
	      end_per_testcase/2 are included in the timetrap time. A TimeFunc
	      function can be used to  set  a  new  timetrap  by  returning  a
	      TimeVal.	It can also be used to trigger a timetrap time-out by,
	      at some point, returning a value other than a TimeVal.  For  de-
	      tails, see section Timetrap Time-Outs in the User's Guide.

	      Tag  require specifies configuration variables that are required
	      by the test case (or init_per_testcase/2 or end_per_testcase/2).
	      If  the required configuration variables are not found in	any of
	      the configuration	files, the test	case is	skipped.  For  details
	      about the	require	functionality, see function ct:require/1,2.

	      If  timetrap or require is not set, the default values specified
	      by suite/0 (or group/1) are used.

	      With userdata, the user can specify any test case-related	infor-
	      mation that can be read by calling ct:userdata/3.

	      Other tuples than	the ones defined are ignored.

	      For  details  about the test case	information function, see sec-
	      tion Test	Case Information Function in the User's	Guide.

       Module:Testcase(Config) -> term() | {skip,Reason} | {comment,Comment} |
       {save_config,SaveConfig}	| {skip_and_save,Reason,SaveConfig} | exit()

	      Types:

		 Config	= SaveConfig = [{Key,Value}]
		 Key = atom()
		 Value = term()
		 Reason	= term()
		 Comment = string()

	      MANDATORY

	      The  implementation  of  a test case. Call the functions to test
	      and check	the result. If something fails,	 ensure	 the  function
	      causes  a	 runtime  error	or call	ct:fail/1,2 (which also	causes
	      the test case process to terminate).

	      Elements from the	Config list can, for  example,	be  read  with
	      proplists:get_value/2 in STDLIB (or the macro ?config defined in
	      ct.hrl).

	      If you decide not	 to  run  the  test  case  after  all,	return
	      {skip,Reason}.  Reason  is  then printed in field	Comment	on the
	      HTML result page.

	      To print some information	in field Comment on  the  HTML	result
	      page, return {comment,Comment}.

	      If  the function returns anything	else, the test case is consid-
	      ered successful. The return value	always	gets  printed  in  the
	      test case	log file.

	      For  details  about  test	 case implementation, see section Test
	      Cases in the User's Guide.

	      For information on save_config and  skip_and_save,  see  section
	      Saving Configuration Data	in the User's Guide.

Ericsson AB		       common_test 1.17			common_test(3)

NAME | DESCRIPTION | TEST CASE CALLBACK FUNCTIONS | EXPORTS

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=common_test&sektion=6&manpath=FreeBSD+12.2-RELEASE+and+Ports>

home | help