Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
include_server(1)	    General Commands Manual	     include_server(1)

NAME
       include_server.py  - conservative approximation of include dependencies
       for C/C++

SYNOPSIS
       include_server --port INCLUDE_SERVER_PORT [OPTIONS]

DESCRIPTION
       include_server.py starts	an include server process.  This  process  an-
       swers  queries  from  distcc(1)	clients	about what files to include in
       C/C++ compilations. The include_server.py command itself	terminates  as
       soon as the include server has been spawned.

       The  INCLUDE_SERVER_PORT	 argument is the name of a socket used for all
       communication between distcc  clients  and  the	include	 server.   The
       pump(1)	command	 is  responsible for creating the socket location, for
       passing it to this script, and for passing it to	all distcc clients via
       the environment variable	named INCLUDE_SERVER_PORT.

       The  protocol  used by the include server uses distcc's RPC implementa-
       tion. Each distcc request consists of (1) the current directory and (2)
       the list	of arguments of	the compilation	command.

       If  the	include	server is able to process the request, then it answers
       the distcc client by sending a list of filepaths.   The	filepaths  are
       those  of  the compressed source	and header files found to be necessary
       for compilation through include analysis.  The list also	comprises sym-
       bolic  links  and even dummy files needed for the compilation server to
       construct an accurate replica of	the parts of the filesystem needed for
       compilation.   In  this	way,  a	needed header file like	/path/foo.h is
       compressed, renamed, and	 stored	 in  a	temporary  location,  such  as
       /dev/shm/tmpiAvfGv.include_server-9368-1/path/foo.h.lzo.	   The	distcc
       client will pass	these files on to a  compilation  server,  where  they
       will be uncompressed and	mounted	temporarily.

       If  the	include	server is not able to process the request, then	it re-
       turns the empty list to the distcc client.

       There are two kinds of failures that relate to the include server.  The
       include	server may fail	to compute the includes	or fail	in other ways,
       see section INCLUDE SERVER SYMPTOMS.  Also, the compilation on the  re-
       move  server  may fail due to inadequacy	of the calculated include clo-
       sure, but then succeed when locally retried, see	 section  DISTCC  DIS-
       CREPANCY	SYMPTOMS.

OPTION SUMMARY
       The following options are understood by include_server.py.

       -dPAT, --debug_pattern=PAT
	      Bit vector for turning on	warnings and debugging
		  1 = warnings
		  2 = trace some functions
	       other powers of two: see	include_server/basics.py.

       -e, --email
	      Send  email  to 'distcc-pump-errors' or if defined, the value of
	      environment variable DISTCC_EMAILLOG_WHOM_TO_BLAME, when include
	      server gets in trouble.  The default is to not send email.

       --email_bound NUMBER
	      Maximal number of	emails to send (in addition to a final email).
	      Default: 3.

       --no-email
	      Do not send email. This is the default.

       --path_observation_re=RE
	      Issue warning message whenever a filename	is resolved to a real-
	      path  that  is  matched  by RE, which is a regular expression in
	      Python syntax.  This is useful for finding out where  files  in-
	      cluded  actually	come  from. Use	RE="" to find them all.	 Note:
	      warnings must be enabled with at least -d1.

       --pid_file FILEPATH
	      The pid of the include server is written to file FILEPATH.  This
	      allows a script such a pump to tear down the include server.

       -s, --statistics
	      Print information	to stdout about	include	analysis.

       --stat_reset_triggers=LIST
	      Flush  stat  caches  when	 the timestamp of any filepath in LIST
	      changes or the filepath comes in or out of existence.  LIST is a
	      colon  separated string of filepaths, possibly containing	simple
	      globs (as	allowed	by Python's  glob  module).  Print  a  warning
	      whenever	such  a	change happens (if warnings are	enabled). This
	      option allows limited exceptions to distcc_pump's	normal assump-
	      tion that	source files are not modified during the build.

       -t, --time
	      Print elapsed, user, and system time to stderr.

       --unsafe_absolute_includes
	      Do  preprocessing	 on the	compilation server even	if includes of
	      absolute filepaths are encountered.  Normally the	include-server
	      will fall	back on	local preprocessing if it detects any absolute
	      includes.	 Thus, this flag is useful for preventing  such	 fall-
	      backs  when  the absolute	includes are a false alarm, either be-
	      cause the	absolute include is discarded during preprocessing  or
	      because  the  absolutely included	file exists on the compilation
	      servers.

	      More precisely,  with  --unsafe_absolute_includes	 absolute  in-
	      cludes  are  ignored  for	 the purposes of gathering the include
	      closure.	Using this option may lead to  incorrect  results  be-
	      cause (1)	the header may actually	be included on the compilation
	      server and it may	not be the same	as on the client, (2) the  in-
	      clude directives of the header are not further analyzed.

	      The  option is useful for	compiling code that has	such hardcoded
	      absolute locations of header files inside	conditional directives
	      (e.g.  "#ifdef") that render the includes	irrelevant.  More pre-
	      cisely, these includes must be eliminated	 during	 preprocessing
	      for  the actual configuration. Then the question of existence of
	      the header file is moot and the  remote  compilation  is	sound.
	      This  is	often  the case	if such	includes are meant for unusual
	      configurations different from the	actual configuration.

       --no_force_dirs
	      Do not force the creation	of all directories used	in an  include
	      path.   May  improve  performance	for some cases,	but will break
	      builds which use include structures like "<foo/../file.h>" with-
	      out including other files	in foo/.

       -v, --verify
	      Verify that files	in CPP closure are contained in	closure	calcu-
	      lated by include processor.

       -w, --write_include_closure
	      Write a .d_approx	file which lists all the included files	calcu-
	      lated by the include server; with	-x, additionally write the in-
	      cluded files as calculated by CPP	to a .d_exact file.

       -x, --exact_analysis
	      Use CPP instead, do not omit system headers files.

INCLUDE	SERVER SYMPTOMS	AND ISSUES
       The most	likely messages	and warnings to	come from the include  proces-
       sor are listed below.

       Preprocessing  locally. Include server not covering: Couldn't determine
       default system include directories
	      To determine the default system header directories, the  include
	      server  runs  the	 compiler once for each	language needed	during
	      its session. This	message	indicates that the compiler  specified
	      to distcc	is not present on the client.

       Preprocessing locally. Include server not covering: Bailing out because
       include server spent more than ...s user	time handling request
	      In uncommon situations, the include server fails to analyze very
	      complicated  macro expressions. The distcc client	will use plain
	      distcc mode.

       Warning:	Filepath must be relative but isn't
	      The include server does not accept absolute filepaths,  such  as
	      /usr/include/stdio.h, in include directives, because there is no
	      guarantee	that this header on  the  compilation  server  machine
	      will  be	the  same  as  that on the client.  The	include	server
	      gives up analyzing the include closure. The distcc client	cannot
	      use pump-mode.

	      To  overcome  this problem in a not always reliable way, set the
	      environment variable  INCLUDE_SERVER_ARGS='--unsafe_absolute_in-
	      cludes' when invoking the	pump script to pass the	--unsafe_abso-
	      lute_includes option to the include server.

       Warning:	Absolute filepath ... was IGNORED
	      The --unsafe_absolute_includes is	in use.	 This  situation  hap-
	      pens under the same circumstances	as when	"Filepath must be rel-
	      ative but	isn't" is issued, but in this case  the	 include  will
	      provide an answer	to the distcc client.

       Warning:	Path '/PATH/FILE' changed/came into existence/no longer	exists
	      These  warnings  are issued when using stat reset	triggers.  Be-
	      cause /PATH/FILE changed,	the include server clears its  caches;
	      the  new version of the file (or the lack	of it) renders the in-
	      clude analysis invalid. This message can usually be ignored;  it
	      does  signify  a	somewhat  precarious use of files by the build
	      system.  It is recommended to fix	the build system so that files
	      are not rewritten.

       Warning:	 For  translation unit ..., lookup of file ... resolved	to ...
       whose realpath is ...
	      This warning occurs with --path_observation_re when a new	 real-
	      path matching a source or	header file is observed.

DISTCC DISCREPANCY SYMPTOMS
       The  interactions  between  the	build  system, distcc, and the include
       server is somewhat complex. When	a distcc commands receives  a  failing
       compilation  from the remote server it retries the compilation locally.
       This section discusses the causes of discrepancies between  remote  and
       local compilation.  These are flagged by	the demotion message:

       __________Warning:  ...	pump-mode compilation(s) failed	on server, but
       succeeded locally.
       __________Distcc-pump was demoted to plain mode.	 See the  Distcc  Dis-
       crepancy	Symptoms section in the	include_server(1) man page.

       The pump	script issues this message at the end of the build. This means
       that for	at least one distcc invocation a local	compilation  succeeded
       after  the  remote compilation failed. Each distcc invocation for which
       such a discrepancy occurred in turn also	issues a message such as:

       Warning:	remote compilation of '...' failed, retried locally and	got  a
       different result.

       The demotion makes subsequent distcc invocations	use plain distcc mode.
       Thus preprocessing will take place on the local machine for the remain-
       der  of	the build.  This technique prevents very slow builds where all
       compilations end	up on the local	machine	after failing remotely.

       Of course, if the local compilations fails after	the  remote  failures,
       then  the distcc	invocation exits with the non-zero status of the local
       compilation. The	error messages printed are also	 those	of  the	 local
       compilation.

       The  fallback behavior for distcc-pump mode to local compilation	can be
       disabled	by setting the	environment  variable  DISTCC_FALLBACK	to  0,
       which  makes  the distcc	command	fail as	soon as	the remote compilation
       has failed. This	setting	is very	useful for debugging  why  the	remote
       compilation  went wrong,	because	now the	output from the	server will be
       printed.

       Next we discuss the possible causes of discrepancies.

       The user	changed	a source or header file	during the build.
	      This yields inconsistent results of course.

       A source	or header file changed during the build.
	      The build	system rewrites	a file.	 For Linux  kernel  2.6,  this
	      happens  for 'include/linux/compile.h' and 'include/asm/asm-off-
	      sets.h'. This condition is fixed by letting the  include	server
	      know  that  it  must  reset its caches when a stat of any	of the
	      files changes. Practically, this is done by gathering the	 files
	      in   a   colon-separated	 list	and   then   setting  the  IN-
	      CLUDE_SERVER_ARGS	environment variable when  invoking  the  pump
	      script,  so that it passes the --stat_reset_triggers option; for
	      example,

		 INCLUDE_SERVER_ARGS="--stat_reset_triggers=include/linux/com-
	      pile.h:include/asm/asm-offsets.h"

       A  header file is potentially included, but does	not exist, and is then
       later included.
	      This occurs when some header foo.h includes another header  file
	      trick.h, but the trick.h file has	not yet	been generated and the
	      inclusion	is actually ignored because  of	 preprocessing	direc-
	      tives.  The  include  server  will  probe	 for  the existence of
	      trick.h, because it overapproximates all	possible  ways	direc-
	      tives  actually  evaluate. The file trick.h is determined	not to
	      exist. If	it is later generated, and then	really included,  then
	      the include server will falsely believe that the file still does
	      not exist.  The solution to this problem is to  make  the	 build
	      system generate trick.h before the first time any	header file is
	      included that makes a syntactic reference	to trick.h

       The include server was started with --unsafe_absolute_includes.
	      This is a	problem	if there are header files locally that do  not
	      exist  remotely  and  that are actually used.  Such includes are
	      often protected by conditional directives	that evaluate so  that
	      are actually used	on only	specific and often uncommon platforms.
	      If you are not compiling for such	a platform,  then  it  may  be
	      correct to use --unsafe_absolute_include.

       The include server has calculated the wrong includes.
	      We do not	know of	such a situation.

EXIT CODES
       The  exit  code	of  include_server.py  is  usually 0. That the include
       server has been started properly	is communicated	through	the  existence
       of the pid_file.

ENVIRONMENT VARIABLES
       DISTCC_EMAILLOG_WHOM_TO_BLAME  The  email  address  to  use for include
       server automated	emails.	The default is 'distcc-pump-errors' (which  is
       an email	address	that probably will not exist in	your domain).

       Additionally,  the  invocation of the compiler may use additional envi-
       ronment variables.

BUGS
       If you think you	have found a distcc bug, please	see the	 file  report-
       ing-bugs.txt  in	 the documentation directory for information on	how to
       report it.

       In distcc-pump mode, the	include	server is  unable  to  handle  certain
       very  complicated  computed includes as found in	parts of the Boost li-
       brary. The include server will time out and distcc will revert to plain
       mode.

       Other known bugs	may be documented on http://code.google.com/p/distcc/

AUTHOR
       The  include  server was	written	by Nils	Klarlund, with assistance from
       Fergus Henderson, Manos Renieris, and Craig Silverstein.	Please	report
       bugs to <distcc@lists.samba.org>.

LICENCE
       You  are	 free  to  use	distcc.	 distcc	(including this	manual)	may be
       copied, modified	or distributed only under the terms of the GNU General
       Public  Licence	version	 2  or later.  distcc comes with absolutely no
       warrany.	 A copy	of the GPL is included in the file COPYING.

SEE ALSO
       distcc(1),     distccd(1),     include_server(1),      and      gcc(1).
       http://code.google.com/p/distcc/	https://ccache.dev/

				  9 June 2008		     include_server(1)

NAME | SYNOPSIS | DESCRIPTION | OPTION SUMMARY | INCLUDE SERVER SYMPTOMS AND ISSUES | DISTCC DISCREPANCY SYMPTOMS | EXIT CODES | ENVIRONMENT VARIABLES | BUGS | AUTHOR | LICENCE | SEE ALSO

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=include_server&sektion=1&manpath=FreeBSD+12.2-RELEASE+and+Ports>

home | help