Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
aeintegratq(1)		    General Commands Manual		aeintegratq(1)

NAME
	aeintegratq - integrate	changes	into projects

SYNOPSIS
	aeintegratq [ option...	 ] project-name...

DESCRIPTION
	The aeintegratq	command	is used	to manage the integrations of one or
	more changes in	one or more projects.  Normally	run via	cron(1)	or
	at(1) with the name of a single	project, aeintegratq will manage all
	operations for integration even	when -Build and	-Test are required on
	multiple architectures.	If a change review is revoked after the	queue
	is running aeintegratq will notice the bad state and silently move on.
	If one or more changes are ended or passed after the queue is running,
	and -loop has been given, aeintegratq will notice the new change[s]
	and integrate them. Additional options allow the integrator full con-
	trol over most aspects of queue	management such	as the order of	inte-
	gration	of multiple changes.

OPTIONS
	The following options are understood:

   Option Summary
	-h	Help, show usage information.

	-H	Help, show usage plus all helpful comment information.

	-a	run on Any machine (normally only IntegrationHost)

	-s	run remote operations via ssh (default rsh)

	-n	No action, just	tell what would	be done.

	-ib s	Specify	(remote) server	for ibegin.

	-ip s	Specify	(remote) server	for ipass.

	-k	Keep the scripts and report files.

	-K	Keep the temp file even	if integration passes.

	-loop	Loop to	process	more changes if	they become available before
		aeintegratq completes.	It will	stop when there	is nothing
		more to	be done.

	-M list	Minimum, run given changes -minimum

	-P list	Precious, do not IFail changes in list,	just stop.

	-R list	Ready, specify order and subset, e.g. -R 29,45

	-S stage
		Pick up	at given stage (diff|build|test|integrate)

	-c change-number
		specify	Change to integrate at Stage

	-p project-name
		specify	single Project name

	NOTE: if custom	options	such as	-P -R -S -c -p are given only a	single
	project	may be integrated since	the options would be meaningless to
	the next project given.

	Some options are present only for testing and investigation.  Note
	that options are rarely	required for normal operations.

   Control Options
	The following options are available for	special	needs.	They control
	the order and disposition of each change awaiting_integration in a
	given project.

	-R[eady] number1,number2...
		This option is used to specify order or	subset to integrate.
		Only those changes listed will be attempted, and in exactly
		the order given. This applies to queue looping if -loop	is
		given. In particular note unless the list includes future
		changes, future	loops will not integrate them.

		Useful if a particular change must go in before	another	for
		some reason.  Or if only integrating one or two	changes	when
		several	are awaiting_integration in the	given project.	A sin-
		gle change may also be specified with the -c[hange] number op-
		tion, which is common for other	aegis commands.	 However the
		-R option allows a list	and if given will override any -c
		given.

	-P[recious] number1,number2...

	-P[recious] all
		This option is used to specify that a particular change	or
		subset of changes should be considered precious.  It neither
		implies	order nor limits the queue run to that subset; it only
		means that the changes should be considered precious.  Note
		that at	least one number (or the keyword all) must be given.

		The concept of precious	means that if the given	change were to
		fail anywhere in the integration process, then the process
		simply stops and leaves	the problem change in the delta	direc-
		tory.  The -IFail would	not actually be	executed.  This	is
		sometimes useful to diagnose a problem which only occurs dur-
		ing integrations.  It is also useful if	the failure is due to
		a transient problem such as unreliable machines	on the net-
		work.  In such a case the integration can be resumed after
		fixing the problem.  See the stage options below.

		If, on the other hand, a precious change makes it through the
		integration process successfully, the option has no effect.

	-M[inimum] number1,number2... or all
		Integrate the given change[s] with the -minimum	option.	 Such
		changes	will be	put on the end of the queue so that the	last
		integrations of	a run will be a	minimum.  This feature allows
		practical use of minimum integrations without requiring	-mini-
		mum on each and	every integration.  See	the section below on
		Minimum	integrations for more information.  If -loop is	given
		any change[s] specified	as minimum will	run at the end of the
		loop in	which they are ready, they will	not be pushed to the
		final loop.

	-ib[server] server-name	or ""

	-ip[server] server-name	or ""
		To specify a remote server on which to run -ibegin or 0	re-
		specively. These options are rarely needed, but	may be useful
		if a project is	hosted on a different file server and has a
		large baseline.	By having the -ibegin run on that server the
		network	traffic	would be greatly reduced and for large
		projects and/or	slow networks can greatly reduce the time re-
		quired for -ibegin.  The option	form of	giving an empty	name
		depends	on the output of df -k giving a	parseable host name.
		If that	is not true on your integration	host architecture, you
		will have to specify the server	name.

	-display display-value or ""
		To specify a valid X display for use during integration	opera-
		tions.

   Stage Options
	The following options allow [re]starting an integration	which has al-
	ready progressed through some stages.  This is useful to deal with
	failed (precious) integrations,	or to finish automatically an integra-
	tion begun by hand.

	-S[tage] diff

	-S[tage] build

	-S[tage] test

	-S[tage] integrate
		Pick up	the integration	at the given stage.  Requires
		-c[hange] number option	to specify the change number.

Advanced Controls
	The integrator may provide for special situations such as operations
	required after -Build and before -Test,	or at the end of a queue run.
	Such capabilities are provided by hooks	and strategies described be-
	low.

   Hooks
	There are a set	of hooks available which are run, if present, before
	and after each stage of	the integration.  They can be used to help en-
	sure that the integrator actually gets some sleep while	managing large
	projects.

	These hooks are	searched for in	the directory $HOME/integration_hooks.
	None need exist; aeintegratq will only pay attention to	any that do
	exist.	Hooks may be any form of executable (script, etc) and are
	called with 2 arguments: project-name change-number.  They run as the
	integrator on the machine from which aeintegratq was started.  They
	are named using	the project name along with a suffix according to what
	place in the integration process you want them to run.

	Note that if a hook for	project	foo exists it is also used for any
	branches under that project.  For example, if you have provided
	foo.pre_ip, it will be run for foo.1 and foo.1.0 as well.  If for some
	reason you want	different (or no) action for project foo.1.0, then you
	would provide foo.1.0.pre_ip which does	what you wish, including noth-
	ing, effectively overriding foo.pre_ip.

	Here is	how to map particular places in	the integration	process	to
	hook suffixes.

		 +--------------------------------------------------+
		 |run at time			       extension    |
		 +--------------------------------------------------+
		 |before attempting -Integrate_Begin   .pre_ib	    |
		 |after	-Integrate_Begin completes     .ib	    |
		 |before attempting -Diff	       .pre_d	    |
		 |after	-Diff completes		       .d	    |
		 |before attempting -Build	       .pre_b	    |
		 |after	-Build completes	       .b	    |
		 |before attempting -Build on <arch>   .pre_<arch>b |
		 |after	-Build on <arch> completes     .<arch>b	    |
		 |before attempting -Test	       .pre_t	    |
		 |after	-Test completes		       .t	    |
		 |before attempting -IPass	       .pre_ip	    |
		 |after	-IPass completes	       .ip	    |
		 |before attempting -IFail	       .pre_if	    |
		 |after	-IFail completes	       .if	    |
		 +--------------------------------------------------+
	The hook program should	exit with 0 if successful or 1 if not.	A non-
	zero exit causes the change being integrated to	fail immediately un-
	less it	was marked precious.

	Note that in most cases	anything done via an .ip hook should probably
	be done	instead	by the ipass_notify command in the project attributes
	file (see aepattr(5) for more information), or the build_time_adjust_-
	notify_command in the project configuration file (see aepconf(5) for
	more information), but the hook	can provide a temporary	way to keep
	going until the	permanent solution can be implemented.

	In addition two	special	hooks, aeintegratq.end and aeintegratq.fail,
	are recognized.	 They are called when aeintegratq finishes a queue
	run.  They are called with 2 arguments like any	other hook (project-
	name change-number) although both the project-name and change-number
	given are of the last change integrated	and may	be less	than useful.

	The .end hook is called	if/when	the queue run is finished and was suc-
	cessful.  Note that this does not mean that no changes failed, only
	that no	queue errors occurred.	This hook might	be used	to invoke an-
	other queue run	on a different project/branch, or possibly even	on the
	same project, if other changes may have	been ended and/or reviewed
	while the first	run was	in progress, see also the -loop	option.	 These
	conditions arise quite often with flex time engineers.	Another	use of
	the .end hook is to automatically build	a new package using the	newly
	integrated project as source.

	If queue errors	were encountered, or a change failed that was marked
	precious, then the .fail hook is called.  An obvious use of that hook
	would be an e-mailed page to the integrator.

   Strategy or Oops-retry
	Sometimes a persistent build problem will plague integrations.	This
	can be very annoying if	it ruins an overnight run, especially if the
	cure is	simple when it happens.	 Examples of this can be timeouts due
	to a busy data server or other transient errors.  Note that this ap-
	plies only to -Build related problems.

	To deal	with such problems the integrator may provide a	strategy
	script specific	to a project.  An executable program should be found
	in $HOME/strategy.<project_name>.  The program will be run as the in-
	tegrator with the delta	directory as current directory.	 The program
	may do any commands necessary to clean up and/or diagnose the error.
	If the script finds the	problem	to be transient	and fix-able, it exits
	successfully (with 0 status) and aeintegratq will re-launch the	-Build
	and log	the re-try.  Otherwise the script should exit with a 1 and the
	change will fail.

Multi-Architecture integrations
	For projects which build and test on multiple architectures, aeinte-
	gratq requires arch_hosts be installed and have	available at least one
	machine	of each	architecture required.	This is	also true if the host
	from which aeintegratq is run is of a different	architecture from the
	target architecture of the project being integrated.

	If you wish to take advantage of multiple architecture automatic inte-
	grations, you can install arch_hosts or	provide	a more simple script
	which will return a machine name according to architecture and job
	type.

Minimum	integrations
	 provides a minimum integration	capability which may be	used for vari-
	ous reasons.  The term minimum may be a	bit counter intuitive.	One
	might think it means to	do the minimum amount of work, however it ac-
	tually means use a minimum of files from the baseline in populating
	the delta directory.  Since no constructed files are put in the	delta
	directory, this	normally leads to actually building everything in the
	project	from sources and, as such, might be considered the most	robust
	of builds.

	Note that any change which removes a file, whether by aerm or aemv,
	results	in an implicit minimum integration.  This is intended to en-
	sure nothing in	the project references the removed file.

	A project may adopt a policy that a product release should be based on
	a minimum integration.	Such a policy may be a reflection of local
	confidence, or lack thereof, in	the project's DMT (Dependency Mainte-
	nance Tool) or build system.  Or it may	be based on a validation
	process	wishing	to make	a simple statement on how the released package
	was produced.

	Another, more transient, reason	a to require a minimum integration
	might be when upgrading	a third	party library, compiler	or maybe even
	OS level.  Any of these	events would signal the	need for a minimum in-
	tegration to ensure everything is rebuilt using	the new	resources.
	This can be done with minimum overhead using the -M option as de-
	scribed	above.

	The cost of a minimum integration varies according to type and size of
	the project.  For very large projects, especially those	building large
	numbers	of binaries, the cost can be large.  However large projects
	also require significant time to fully populate	the delta directory.
	A minimum integration only copies those	files under aegis control,
	skipping all "produced"	files.	In the case where a file upon which
	everything depends is changed, everything will be built	anyway so the
	copy of	the already built files	is a waste of time.  This means	that
	sometimes a minimum can	be as cheap as a normal	integration.

Manual Tests
	 allows	tests to be defined as manual which may	be necessary if	the
	test requires human interaction	or some	transient resource. Such tests
	can be problematic for automatic integrations and generally must have
	some means to pass without running during integrations.	For this, and
	other, reasons most sites seek to avoid	manual tests. There are	a num-
	ber of ways to code a test such	that it	will pass automatically	during
	integrations. Just one example for shell script	tests might be:

	CSTATE=`aesub -p $AEGIS_PROJECT	-c $AEGIS_CHANGE '${state}'`
	if [ "$CSTATE" = "being_integrated" ]
	then
	    echo "`basename $0`	passes during integration"
	    exit 0
	fi

Optional Support Programs
	There are some programs	which aeintegratq will use if they are in-
	stalled.

	o arch_hosts was mentioned previously.	It is optional only if your
	projects and your  file	server are of a	single architecture.

	o aelogres may enhance the information provided	in -IFail entries.
	Normally all you get is	the last 10 lines of the log file, which is
	not bad	if tests fail, but can be terrible for failed builds.  If you
	provide	a program named	aelogres which knows how to extract a better
	succinct report	of problems, the output	of that	program	will be	used
	instead	of the simple tail.  It	is called with a -i option.

	o sound_all_machines, if available, will be called when	integrations
	either pass or fail.  It can be	helpful	to announce the	fact that an
	integration has	finished.  If it passed, developers will probably want
	to do an aed to	bring their changes up to date.	 The audio announce-
	ment provides another timely hint.

	The sound files	are searched for in the	/usr/local/com/sounds direc-
	tory.  They will have endings of _pass and _fail according to the re-
	sults of a given attempt.  Two sound files are required: integra-
	tion_pass and integration_fail.	 Others	will be	used if	provided to
	customize the sounds so	that each developer may	have one or more per-
	sonal sounds.  If a file named _developer__pass	is located, it will be
	used.  If a set	of files exist named _developer_pass.[0-9] they	will
	be used	in random sequence.  The same search rule applies to _fail
	sets.  The sound_all_machines program may use a	host list and play the
	sound file on each machine or, assuming	that other audio capabilities
	exist, might do	any form of announcement desired.

EXIT STATUS
	The aeintegratq	command	will exit with a status	of 1 on	any error.
	The aeintegratq	command	will only exit with a status of	0 if there are
	no errors.

ENVIRONMENT VARIABLES
	See aegis(1) for a list	of environment variables which may affect this
	command.  See aepconf(5) for the project configuration file's
	project_specific field for how to set environment variables for	all
	commands executed by Aegis.

FILES
	Control	files are searched for in the $HOME directory.	They are named
	strategy.<project>, They need not exist	if no special action is	neces-
	sary.

	The hook scripts are searched for in the $HOME/integration_hooks di-
	rectory.  They are named <project>.<stage>.  Also aeintegratq.end and
	aeintegratq.fail.  These hooks also need not exist if no special ac-
	tion is	desired.

COPYRIGHT
	aeintegratq version 4.25.D510
	Copyright (C) 1998-2005	Endocardial Solutions, Inc.

	The aeintegratq	program	comes with ABSOLUTELY NO WARRANTY; This	is
	free software and you are welcome to redistribute it under certain
	conditions;

Reference Manual		     Aegis			aeintegratq(1)

NAME | SYNOPSIS | DESCRIPTION | OPTIONS | Advanced Controls | Multi-Architecture integrations | Minimum integrations | Manual Tests | Optional Support Programs | EXIT STATUS | ENVIRONMENT VARIABLES | FILES | COPYRIGHT

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=aeintegratq&sektion=1&manpath=FreeBSD+12.1-RELEASE+and+Ports>

home | help