Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
aegis -New_Test(1)	    General Commands Manual	    aegis -New_Test(1)

NAME
	aegis new test - add a new test	to a change

SYNOPSIS
	aegis -New_Test	[ option...  ][	filename...  ]
	aegis -New_Test	-List [	option...  ]
	aegis -New_Test	-Help

DESCRIPTION
	The aegis -New_Test command is used to add a new test to a change.  A
	new file is created in the development directory.

	New tests default to "automatic" unless	otherwise specified.

   File	Name Interpretation
	The aegis program will attempt to determine the	project	file names
	from the file names given on the command line.	All file names are
	stored within aegis projects as	relative to the	root of	the baseline
	directory tree.	 The development directory and the integration direc-
	tory are shadows of this baseline directory, and so these relative
	names apply here, too.	Files named on the command line	are first con-
	verted to absolute paths if necessary.	They are then compared with
	the baseline path, the development directory path, and the integration
	directory path,	to determine a baseline-relative name.	It is an error
	if the file named is outside one of these directory trees.

	The -BAse_RElative option may be used to cause relative	filenames to
	be interpreted as relative to the baseline path; absolute filenames
	will still be compared with the	various	paths in order to determine a
	baseline-relative name.

	The relative_filename_preference in the	user configuration file	may be
	used to	modify this default behavior.  See aeuconf(5) for more infor-
	mation.

   Test	Filename Generation
	You may	choose your own	filename for a test, by	specifying it on the
	command	line.

	If no filename is specified on the command line, a test	filename is
	automatically generated.  This is controlled by	the new_test_filename
	field of the project configuration file	(see aepconf(5)	for more in-
	formation.  All	automatically generated	test filenames within a
	project	are numbered uniquely.	The default pattern for	new test file-
	names is "test/XX/tXXXX[am].sh", where XX is the first 2 digits	of the
	test number, XXXX is the whole test number, and	[am] is	a for auto-
	matic tests and	m for manual tests.

   Modifying Tests
	Tests may be modified in future	by adding them to a change with	the
	aecp(1)	command.  Tests	are treated just like any other	source file,
	and are	subject	to the same process.

   File	Templates
	When a new file	is created in the development directory	the project
	config file is searched	for a template for the new file.  If a tem-
	plate is found,	the new	file will be initialized to the	template, oth-
	erwise it will be created empty.  See aepconf(5) for more information.

	The simplest form is to	use template files, such as
		file_template =
		[
		    {
			pattern	= [ "*.c" ];
			body = "${read_file ${source template/c	abs}}";
		    },
		    {
			pattern	= [ "test/*/.sh" ];
			body = "${read_file ${source template/test abs}}";
		    },
		];
	As you can see,	the template files are part of the project source, so
	you can	add the	appropriate copyright notices, and wrappers, etc.  The
	$source	substitution locates them, if they are not part	of the current
	change (and they usually are not).

	The template files themselves contain substitutions.  The $filename
	substitution is	available, and contains	the name of the	file being
	created.  This can be manipulated in various ways when constructing
	the appropriate	file contents.	See aesub(5) for more information
	about substitutions.

	It is also possible to run a command to	create the new file.  You can
	do this	instead	of specifying a	body string, viz:
		file_template =
		[
		    {
			pattern	= [ "*"	];
			body_command = "perl ${source template.pl abs} $filename";
		    },
		];
	The command is run with	a current directory set	to the top of the de-
	velopment directory.  It is an error if	the command fails to create
	the file.  You can mix-and-match the two techniques, body string and
	body_command, if you want.

	Be careful to make sure	that the test filename template	pattern
	matches	the new_test_filename field.

   File	Name Limitations
	There are a number of controls available to limit the form of project
	file names.  All of these controls may be found	in the project config-
	uration	file, see aepconf(5) for more information.  The	most signifi-
	cant are briefly described here:

	maximum_filename_length	= integer;
		This field is used to limit the	length of filenames.  All new
		files may not have path	components longer than this.  Defaults
		to 255 if not set.  For	maximum	portability you	should set
		this to	14.

	posix_filename_charset = boolean;
		This field may be used to limit	the characters allowed in
		filenames to only those	explicitly allowed by POSIX.  Defaults
		to false if not	set, meaning whatever your operating system
		will tolerate, except white space and high-bit-on characters.
		For maximum portability	you should set this to true.

	dos_filename_required =	boolean;
		This field may be used to limit	filenames so that they conform
		to the DOS 8+3 filename	limits and to the DOS filename charac-
		ter set.  Defaults to false if not set.

	windows_filename_required = boolean;
		This field may be used to limit	filenames so that they conform
		to the Windows98 and WindowsNT filename	limits and character
		set.  Defaults to false	if not set.

	shell_safe_filenames = boolean;
		This field may be used to limit	filenames so that they do not
		contain	shell special characters.  Defaults to true if not
		set.  If this field is set to false, you will need to use the
		${quote} substitution around filenames in commands, to ensure
		that filenames containing shell	special	characters do not have
		unintended side	effects.  Weird	characters in filenames	may
		also confuse your dependency maintenance tool.

	allow_white_space_in_filenames = boolean;
		This field may be used to allow	white space characters in file
		names.	This will allow	the following characters to appear in
		file names: backspace (BS, \b, 0x08), horizontal tab (HT, \t,
		0x09), new line	(NL, \n, 0x0A),	vertical tab (VT, \v, 0x0B),
		form feed (FF, \f, 0x0C), and carriage return (CR, \r, 0x0D).
		Defaults to false if not set.

		Note that this field does not override other file name fil-
		ters.  It will be necessary to explicitly set shell_safe_-
		filenames = false as well.  It will be necessary to set	dos_-
		filename_required = false (the default)	as well.  It will be
		necessary to set posix_filename_charset	= false	(the default)
		as well.

		The user must take great care to use the ${quote} substitution
		around all file	names in commands in the project configura-
		tion.  And even	then, substitutions which expect a space sepa-
		rated list of file names will have undefined results.

	allow_non_ascii_filenames = boolean;
		This field may be used to allow	file names with	non-ascii-
		printable characters in	them.  Usually this would mean a UTF8
		or international charset of some kind.	Defaults to false if
		not set.

		Note that this field does not override other file name fil-
		ters.  It will be necessary to explicitly set shell_safe_-
		filenames = false as well.  It will be necessary to set	dos_-
		filename_required = false (the default)	as well.  It will be
		necessary to set posix_filename_charset	= false	(the default)
		as well.

	filename_pattern_accept	= [ string ];
		This field is used to specify a	list of	patterns of acceptable
		filenames.  Defaults to	"*" if not set.

	filename_pattern_reject	= [ string ];
		This field is used to specify a	list of	patterns of unaccept-
		able filenames.

	Please Note: Aegis also	consults the underlying	file system, to	deter-
	mine its notion	of maximum file	size.  Where the file system's maximum
	file size is less than maximum_filename_length,	the filesystem wins.
	This can happen, for example, when you are using the Linux UMSDOS file
	system,	or when	you have an NFS	mounted	an ancient V7 filesystem.
	Setting	maximum_filename_length	to 255 in these	cases does not alter
	the fact that the underlying file systems limits are far smaller (12
	and 14,	respectively).

	If your	development directories	(or your whole project)	is on filesys-
	tems with filename limitations,	or a portion of	the heterogeneous
	builds take place in such an environment, it helps to tell Aegis what
	they are (using	the project config file's fields) so that you don't
	run into the situation where the project builds	on the more permissive
	environments, but fails	with mysterious	errors in the more limited en-
	vironments.

	If your	development directories	are routinely on a Linux UMSDOS
	filesystem, you	would probably be better off setting dos_filename_re-
	quired = true, and also	changing the development_directory_template
	field.	Heterogeneous development with various Windows environments
	may also require this.

   Changing the	Type of	a File
	If you want to change the type of a file (say, from a test to a	source
	file, or vice versa) you could do it as	two changes, by	first using
	aerm(1)	in one change and then using aenf(1) or	aent(1)	in a second
	change,	or you can combine both	steps in the same change.  Remember to
	use the	aerm -nowhiteout option	or you will get	a most peculiar	new
	file template.

   Notification
	The new_test_command in	the project config file	is run,	if set.	 The
	project_file_command is	also run, if set, and if there has been	an in-
	tegration recently.  See aepconf(5) for	more information.

TEST PROCESS
	Each change is required	to be accompanied by tests, and	those tests
	are required to	be run against the built development directory,	and
	they must pass.	 This ensures that new functionality is	accompanied by
	tests to verify	its correctness, and bug fixes are accompanied by
	tests which confirm that the bug has been fixed.

   Regression Tests
	Tests are treated as any other source file, and	are maintained in the
	baseline and history with all other source files.  The tests which
	must accompany every change accumulate in the project baseline,	pro-
	viding a definition of correct function	for the	baseline.  These accu-
	mulated	tests may be executed using an "aegis -REGression" command, to
	verify that the	project	will not "regress" as a	result of a change.

   Baseline Tests
	Bug fixes are required to have their tests fail	against	the project
	baseline (in contrast to the development directory).  This ensures
	that the test actually demonstrates the	bug in the baseline, as	well
	as demonstrating that it is fixed by the change.  New functionality
	trivially fails	against	the baseline, and so aegis does	not attempt to
	guess if a test	is a bug fix test or new functionality test, it	simply
	requires tests to fail against the baseline.

	This requirement applies both to new tests being created by a change
	and also to tests which	have been copied into a	change for modifica-
	tion.

   Reviewing Tests
	Reviewers may be confident that	aegis has enforced the test require-
	ments; that a change must have tests, that the change must build, that
	the tests pass against the development directory, and that the tests
	fail against the baseline.  These conditions are enforced by aede(1)
	and the	change will not	be advanced to the being reviewed state	until
	these conditions are met.  Reviewers should thus review	tests for com-
	pleteness of coverage of the code in the change, and insensitivity to
	changes	in the execution environment (e.g. not date sensitive).	 Re-
	viewers	should also use	"aegis -list change_details" to	verify that a
	change does or does not	have testing exemptions.

   Exemptions
	Various	test exemptions	may be granted by project administrators, see
	aepa(1)	and aepattr(5) for more	information.  Copying tests into a
	change,	or adding new tests to a change, may cancel those exemptions.

TEST CORRELATIONS
	The "aegis -Test -SUGgest" command may be used to have aegis suggest
	suitable regression tests for your change, based on the	source files
	in your	change.	 This automatically focuses testing effort to relevant
	tests, reducing	the number of regression tests necessary to be confi-
	dent that you have not introduced a bug.

	The test correlations are generated by the "aegis -Integrate_Pass"
	command, which associates each test in the change with each source
	file in	the change.  Thus, each	source file accumulates	a list of
	tests which have been associated with it in the	past.  This is not as
	exact as code coverage analysis, but is	a reasonable approximation in
	practice.

	The aecp(1) and	aenf(1)	commands are used to associate files with a
	change.	 While they do not actively perform the	association, these are
	the files used by aeipass(1) and aet(1)	to determine which source
	files are associated with which	tests.

   Test	Correlation Accuracy
	Assuming that the testing correlations are accurate and	that the tests
	are evenly distributed across the function space, there	will be	a less
	than 1/number chance that a relevant test has not been run by the
	"aegis -Test -SUGgest number" command.	A small	amount of noise	is
	added to the test weighting, so	that unexpected	things are sometimes
	tested,	and the	same tests are not run every time.

	Test correlation accuracy can be improved by ensuring that:

	o Each change should be	strongly focused, with no gratuitous file in-
	  clusions.  This avoids spurious correlations.

	o Each item of new functionality should	be added in an individual
	  change, rather than several together.	 This strongly correlates
	  tests	with functionality.

	o Each bug should be fixed in an individual change, rather than	sev-
	  eral together.  This strongly	correlates tests with functionality.

	o Test correlations will be lost if files are moved.  This is because
	  correlations are by name.

	The best way for tests to correlate accurately with source files is
	when a change contains a test and exactly those	files relating to the
	functionality under test.  Too many spurious files will	weaken the
	usefulness of the testing correlations.

OPTIONS
	The following options are understood;

	-AUTOmatic
		This option may	be used	to specify automatic tests.  Automatic
		tests require no human assistance.

	-BAse_RElative
		This option may	be used	to cause relative filenames to be con-
		sidered	relative to the	base of	the source tree.  See aeu-
		conf(5)	for the	corresponding user preference.

	-CUrrent_RElative
		This option may	be used	to cause relative filenames to be con-
		sidered	relative to the	current	directory.  This is usually
		the default.  See aeuconf(5) for the corresponding user	pref-
		erence.

	-Change	number
		This option may	be used	to specify a particular	change within
		a project.  See	aegis(1) for a complete	description of this
		option.

	-Edit	Edit the new test files	one they have been created.  (This
		avoids the copy-and-paste step required	to edit	the new	test
		script when it has an automatically generated file name.)

	-Help
		This option may	be used	to obtain more information about how
		to use the aegis program.

	-List
		This option may	be used	to obtain a list of suitable subjects
		for this command.  The list may	be more	general	than expected.

	-MANual	This option may	be used	to specify manual tests.  Manual tests
		require	some human intervention,  e.g.:	confirmation of	some
		screen behavior	(X11, for instance), or	some user action, "un-
		plug ethernet cable now".

	-Not_Logging
		This option may	be used	to disable the automatic logging of
		output and errors to a file.  This is often useful when	sev-
		eral aegis commands are	combined in a shell script.

	-Output	filename
		This option may	be used	to specify a filename which is to be
		written	with the automatically determined test file name.
		Useful for writing scripts.

	-Project name
		This option may	be used	to select the project of interest.
		When no	-Project option	is specified, the AEGIS_PROJECT	envi-
		ronment	variable is consulted.	If that	does not exist,	the
		user's $HOME/.aegisrc file is examined for a default project
		field (see aeuconf(5) for more information).  If that does not
		exist, when the	user is	only working on	changes	within a sin-
		gle project, the project name defaults to that project.	 Oth-
		erwise,	it is an error.

	-TEMplate
		This option may	be used	to specify that	a new file template
		should be used,	even if	the file already exists.

	-No_TEMplate
		This option may	be used	to specify that	a new file template
		should not be used, even if the	file does not exist (any empty
		file will be created).

	-TERse
		This option may	be used	to cause listings to produce the bare
		minimum	of information.	 It is usually useful for shell
		scripts.

	-Universal_Unique_IDentifier string
		This option may	be used	to set the UUID	of a file.

	-Not_Universal_Unique_IDentifier
		This option may	be used	to require that	the file is created
		without	an UUID.  The aeipass-option:assign-file-uuid is set
		to false for the file to avoid automatic UUID assignment when
		aeipass(1) is invoked.

	-Verbose
		This option may	be used	to cause aegis to produce more output.
		By default aegis only produces output on errors.  When used
		with the -List option this option causes column	headings to be
		added.

	-Wait	This option may	be used	to require Aegis commands to wait for
		access locks, if they cannot be	obtained immediately.  De-
		faults to the user's lock_wait_preference if not specified,
		see aeuconf(5) for more	information.

	-No_Wait
		This option may	be used	to require Aegis commands to emit a
		fatal error if access locks cannot be obtained immediately.
		Defaults to the	user's lock_wait_preference if not specified,
		see aeuconf(5) for more	information.

	See also aegis(1) for options common to	all aegis commands.

	All options may	be abbreviated;	the abbreviation is documented as the
	upper case letters, all	lower case letters and underscores (_) are op-
	tional.	 You must use consecutive sequences of optional	letters.

	All options are	case insensitive, you may type them in upper case or
	lower case or a	combination of both, case is not important.

	For example: the arguments "-project", "-PROJ" and "-p"	are all	inter-
	preted to mean the -Project option.  The argument "-prj" will not be
	understood, because consecutive	optional characters were not supplied.

	Options	and other command line arguments may be	mixed arbitrarily on
	the command line, after	the function selectors.

	The GNU	long option names are understood.  Since all option names for
	aegis are long,	this means ignoring the	extra leading '-'.  The	"--op-
	tion=value" convention is also understood.

RECOMMENDED ALIAS
	The recommended	alias for this command is
	csh%	alias aent 'aegis -nt \!* -v'
	sh$	aent(){aegis -nt "$@" -v}

ERRORS
	It is an error if the change is	not in the being developed state.
	It is an error if the change is	not assigned to	the current user.

EXIT STATUS
	The aegis command will exit with a status of 1 on any error.  The
	aegis command will only	exit with a status of 0	if there are no	er-
	rors.

ENVIRONMENT VARIABLES
	See aegis(1) for a list	of environment variables which may affect this
	command.  See aepconf(5) for the project configuration file's
	project_specific field for how to set environment variables for	all
	commands executed by Aegis.

SEE ALSO
	aecp(1)	copy an	existing test into a change

	aedb(1)	begin development of a change

	aentu(1)
		remove a new test from a change

	aerm(1)	remove an existing test	as part	of a change

	aet(1)	run tests

	aeuconf(5)
		user configuration file	format

COPYRIGHT
	aegis version 4.25.D510
	Copyright (C) 1991, 1992, 1993,	1994, 1995, 1996, 1997,	1998, 1999,
	2000, 2001, 2002, 2003,	2004, 2005, 2006, 2007,	2008, 2009, 2010,
	2011, 2012 Peter Miller

	The aegis program comes	with ABSOLUTELY	NO WARRANTY; for details use
	the 'aegis -VERSion License' command.  This is free software and you
	are welcome to redistribute it under certain conditions; for details
	use the	'aegis -VERSion	License' command.

AUTHOR
	Peter Miller   E-Mail:	 pmiller@opensource.org.au
	/\/\*		  WWW:	 http://miller.emu.id.au/pmiller/

Reference Manual		     Aegis		    aegis -New_Test(1)

NAME | SYNOPSIS | DESCRIPTION | TEST PROCESS | TEST CORRELATIONS | OPTIONS | RECOMMENDED ALIAS | ERRORS | EXIT STATUS | ENVIRONMENT VARIABLES | SEE ALSO | COPYRIGHT | AUTHOR

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=aent&sektion=1&manpath=FreeBSD+12.0-RELEASE+and+Ports>

home | help