Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
MAKEPP_SCANNING(1)		    Makepp		    MAKEPP_SCANNING(1)

NAME
       makepp_scanning -- How makepp finds include files and other hidden
       dependencies

DESCRIPTION
       Makepp can determine additional dependencies or targets for certain
       commands	that it	knows something	about.	This is	especially important
       for C/C++ compilation, where it is too error-prone to list manually all
       of the include files that a given source	file depends on.  By looking
       at the compilation command and the source files themselves, makepp is
       able to determine accurately which object files need to be rebuilt when
       some include file changes.

       Example:	Given a	rule

	   foo.o:		       # Usually %.o: %.c, just	for illustration
	       time -p /bin/libtool -bar /usr/bin/cc -c	-I somewhere foo.c

       makepp knows that "time"	and "libtool" must be skipped and that "cc" is
       the actual command to be	parsed here.  It understands that foo.c	is the
       input file and thus a dependency	of this	rule.  Moreover	it will	scan
       that file looking for include statements, also in directory somewhere,
       because it understood the command options.

       Actually	there are three	steps to what is historically known as
       scanning:

       1.  The rule action is split into lines (continuation lines count as
	   one).  Each line (except builtins and Perl blocks) is lexically
	   analyzed as one or more Shell commands.  Redirections are
	   recognized as inputs	or outputs to this rule.  The first word of
	   each	command	is looked up (with its directory part but, if not
	   found, again	without	it) to find a parser for it.  These become
	   optional dependencies, they are built if possible, but ignored if
	   not found, as makepp	can't know which part of a complex command is
	   actually run.

	   Commands in backquotes are analyzed but not executed.  (Often
	   execution is	important, but this would be a major interference by
	   makepp.)  It	is better style	to avoid them.	Instead	have makepp
	   run the command at most once	by assigning it	in this	special	way:

	       XYZFLAGS	;= $(shell pkg-config --cflags xyz)

	   Currently there is only one lexer class, which understands Bourne
	   Shell.  To better handle C Shell or "command.com", subclasses might
	   be created.	However, much syntax is	similar	enough to not warrant
	   this.  Get in touch if you want to contribute either.

       2.  For known commands the corresponding	command	parser (also referred
	   to just as parser) analyzes the important options and arguments.
	   The available ones are described below.

	   Even	if no specialized parser was found, the	generic	one makes the
	   command executable an input of this rule.  You can change that with
	   the --no-path-executable-dependencies command option.

       3.  If the parser recognized any	input files, they get sent to the
	   scanner chosen by the parser.  It finds further inputs by looking
	   for "#include" or comparable	statements.

	   This	is the most expensive step.  All the results get cached	to
	   avoid repeating it unnecessarily.

       If makepp thinks	it's compiling a C/C++ source but can't	find a parser,
       it will give a warning message to let you know.	This usually means
       that you	buried your compiler command too deeply	in the action for
       makepp to find it.  For example,	I have seen rules like this:

	   %.o:	%.c
	       @echo Compiling $< now; obscure_wrapper gcc -c $< $(CFLAGS) -o $@

       The first words of the actions here are "echo" and "obscure_wrapper",
       for which there are no parsers, so makepp will not scan for include
       files in	this case.  You	can ignore the prefixed	command	by:

	   register-parser obscure_wrapper skip-word

       The following sections document the built in parsers and	scanners.  In
       the name	you can	use "-"	interchangeably	with "_".

SCANNERS (PARSERS)
       The various scanners must be chosen by a	command	parser,	which is given
       in parentheses:

   C/C++ compilation (c-compilation, gcc-compilation)
       The C/C++ scanner, handles both languages indifferently.	 In fact it
       looks only at preprocessor statements, so it can	be used	for quite a
       few languages.  The parser that activates it has	a special variant for
       gcc's many options, which gets chosen if	the command name includes the
       string "gcc" or g++.  If	compilers for other languages with C
       preprocessor use	the same options as the	C compiler (at least "-I")
       then this parser	works fine.

       It looks	at the command for "-Idir" options specifying the include path
       or "-Ldir" options specifying the link path.  It	then scans any source
       files for "#include" directives,	and also looks at the command line to
       see if there are	any source files or libraries mentioned	which are not
       listed as dependencies.	It recognizes these by their extension.

       This scanner gives a warning message if files included with
       "#include "file.h"" are not found, or not buildable by makepp, in the
       include path, or	in the directory containing the	file which is
       "#includ"ing, or	in /usr/include.  No warning is	given if a file
       included	with "#include <file.h>" is not	found.	Makepp assumes it is
       in some system include directory	that the compiler knows	about, and
       that files in system include directories	won't change.

       In addition, files in /usr/include, /usr/local/include,
       /usr/X11R6/include, and any other directory which is not	writable are
       not scanned to see what they include.  Makepp assumes that these	files
       won't change.  (If you're running as root, the writability test is
       performed with the UID and GID of the directory you ran makepp from.
       This is so compiling a program as an ordinary user and then doing
       "makepp install"	as root	won't cause extra directories to be scanned.)

       This is a fairly	simple-minded scanner.	It will	get confused if	you do
       things like this:

	   #ifdef INCLUDE_THIS
	   #include "this.h"
	   #endif

       because it doesn't know about preprocessor conditionals.	 This is
       usually harmless; it might cause	additional extra files to be labeled
       as dependencies (occasionally causing unnecessary rebuilds), or else it
       might cause makepp to warn that the include file	was not	found.	You
       can either ignore the warning messages, or put an empty file "this.h"
       out there to shut makepp	up.

       If your compiler	has a funny name, you can say either of

	   register-parser obscure_c_compiler c-compilation
	   register-parser obscure_gcc_alias gcc-compilation

   Embedded SQL	C/C++ compilation (esql-compilation)
       These commands, which come with the various databases, preprocess
       special sections	in otherwise C/C++-like	sources, and produce C/C++
       headers and sources.  This finds	EXEC SQL INCLUDE "filename" or
       $INCLUDE	"filename" directives.

       These preprocessors are recognized: Altibase APRE*C/C++ (apre),
       CASEMaker DBMaker (dmppcc), Firebird / InterBase	(gpre),	IBM DB2	(db2
       precompile, db2 prep) & Informix	ESQL/C (esql), Ingres (esqlc), Mimer
       (esql), Oracle (proc), PostgreSQL (ecpg)	& YARD (yardpc).  If your
       preprocessor is not recognized, you can say

	   register-parser obscure_esqlc_preprocessor esql-compilation

       This will however only handle the style common to Informix and others:
       Command arguments ending	in ".ec" are files to be scanned, "-I" defines
       the include path	and EXEC SQL INCLUDE directives	without	a suffix get
       ".h" appended.

   Swig	(swig)
       Swig (Simplified	Wrapper	and Interface Generator, http://www.swig.org/)
       is a program that converts a C/C++ header file into the wrapper
       functions needed	to make	your code callable from	a variety of other
       languages, such as Perl,	Python,	Tcl, C#, Ruby, OCaml, and probably
       some others that	I don't	know about.

       Makepp understands and parses the swig command line, looking for	"-I"
       and "-l"	options.  It also knows	how to scan swig's interface
       definition files	(.i files) looking for %include, %import, and also
       "#include" if "-includeall" is in effect.

       If your swig has	a funny	name, you can say

	   register-parser obscure_swig_alias swig

   Vera	and Verilog (vcs_compilation)
       If you design hardware, this will come in handy.

   Ignorable wrappers (skip-word, shell)
       Makepp recognizes the following command words and many more and skips
       over them appropriately in in its search	for the	correct	scanner:
       "condor_compile", "distcc", "ignore_error", "libtool", "noecho"
       "purify".

       There is	a variant of this which	finds the nested commands in "sh -c
       'command1; command2'".

       If you have more	such commands, you can say

	   register-parser command skip-word

       Libtool

       Libtool is a very clever	compilation system that	greatly	simplifies
       making shared libraries by hiding all the system-dependent details away
       in a shell script.  The only difficulty is that the library binary
       files are not actually stored in	the same directory as the output
       file--libtool actually creates a	subdirectory, ".libs", which contains
       the real	files.	This is	ordinarily not a problem, but makepp has to
       know where the real binaries are	if it is to link them in from a
       repository.  At the moment, libtool libraries (".la" files) are not
       linked in from repositories; they are always rebuilt if needed.	Also,
       makepp at the moment is not able	to use the dependency information that
       is stored inside	the ".la" file itself.	This will hopefully change
       soon.

   Suppressing the scan	(none)
       Sometimes you may not want a rule or a certain command to be parsed.
       You can turn off	parsing	and thereby scanning with

	   register-parser cc none

RELATED	OPTIONS
   Quickscan and smartscan
       The ":quickscan"	and ":smartscan" rule options, if applicable, affect
       the way that files are scanned.

       In ":quickscan" mode (the default), all include directives are assumed
       active. This allows for very efficient scanning.

       In ":smartscan" mode, an	attempt	is made	to interpret macros and
       expressions so that inactive include directives are ignored.  For
       example,	the executable produced	by compiling the following C program
       ought not to depend on foo.h:

	   #if 0
	   #include "foo.h"
	   #endif
	   int main() {	return 0; }

CUSTOM SCANNERS
       You can specify your own	parser either in a rule	option like ":parser
       foo", or	by using the "register_parser" or "register_command_parser"
       statements.

       Either way, as described	under "register_parser", there you must
       directly	or indirectly (via a class) specify a function that creates a
       parser object.  This object will	usually	create a scanner object	for
       files, and feed it with its findings from the command line options.
       These two objects will call the parser's	"add_*_dependency" methods
       which forward the information to	the somewhat more complicated
       "Mpp::Lexer::add_*_dependency" utility functions.

       However your parser function can	also do	this work itself for simple
       cases.  There are a couple of special return values if this function
       doesn't return a	parser object:

       "undef"
	   The scan info is not	cacheable and must be recalculated next	time
	   the rule's target needs to be built.

       "p_none,	p_skip_word" or	"p_shell"
	   These are in	fact numeric constants,	which tell the lexer to	do the
	   work	of these pseudo-parsers.

       any reference, e.g. "\1"
	   This	is equivalent to returning a parser object of the
	   "Mpp::CommandParser"	base class, which will only additionally make
	   the command executable itself a dependency.

       In most cases, objects of type "Mpp::CommandParser" should instantiate
       at least	one object of type "Mpp::Scanner".  The	"Mpp::Scanner" base
       class takes care	of the distinction between quickscan and smartscan.
       Note that the behavior of "Mpp::Scanner"	can be markedly	affected by
       this distinction, but that should be transparent	to the derived class
       if it is	well-formed.  New derived "Mpp::Scanner" classes ought to be
       tested in both modes.

       If you write your own "Mpp::Scanner" class, you should also base	your
       rescanning decision on the build	info "RESCAN".	This gets set by
       "makeppreplay" after signing files without scanning.  So	despite	the
       signatures being	consistent, a rescan is	still necessary.  If your
       "Mpp::Scanner" uses the inherited "scan_file1" method, you're probably
       fine.

       For more	details, refer to the respective class documentation.  For
       examples, see "Mpp::CommandParser::Gcc" and "Mpp::CommandParser::Vcs".
       Look at the "p_"	functions in Mpp/Subs.pm which get aliased into	their
       respective classes as "factory" when loaded.

   Caching scanner info
       If the all of the scanner's important side effects are effected through
       calls to	methods	of the "Mpp::CommandParser" base class,	then those
       side effects can	be cached in the build info file, so that they can be
       played back by a	subsequent invocation of makepp	without	doing all of
       the costly scanning work.  This can save	quite a	bit of time,
       especially in smartscan mode.

       If the scanner has other	important side effects,	then it	should call
       the "Rule" object's mark_scaninfo_uncacheable method.  Otherwise, the
       scanner info retrieved from the build info may be inaccurate, causing
       the build result	possibly to be incorrect.  This	method is called
       automatically when a value from the %parsers hash does not return an
       object of type "Mpp::CommandParser", or when the	parser is specified
       with a rule option and the "p_*"	routine	does not return	an object of
       type "Mpp::CommandParser".

       Cached scan info	is invalidated using criteria similar to those used
       for determining when the	target is out of date.	Similarly, it can be
       retrieved from a	repository using criteria similar to those used	for
       determining when	a target can be	linked in from a repository.

       You can force makepp to ignore the cached scanner info with the
       "--force-rescan"	option.	 This is useful	when a broken scanner may have
       caused incorrect	scanner	info to	be cached.

   Ad Hoc Scanner
       Often you will have just	one or few files which contain dependency
       information.  You don't want to write this into a makefile redundantly
       (since redundancy later often leads to inconsistencies when one update
       gets forgotten).	 But you also don't want to write a Mpp::Scanner?  As
       a workaround you	can generate an	include	file on	the fly.  For example
       Qt has .qrc files which can look	like:

	   <RCC>
	     <qresource	prefix="...">
	       <file>abc</file>
	       <file>xyz</file>
	       ...

       If you adhere to	the above layout, you can transform the	relevant lines
       into a makepp include file, which gets automatically created by being
       included.

	   %.qrc.makepp: %.qrc
	       &grep 's!<RCC>\n!$(stem).cc:! ||	s! *<file>! ! && s!</file>\n!!'	$(input) -o $(output)

	   include $(wildcard *.qrc)   # .makepp is appended automatically

       The drawback is that you	begin building while reading the makefile.  So
       the --stop-after-loading	command	option will not	be very	useful.

perl v5.32.1			  2012-02-07		    MAKEPP_SCANNING(1)

NAME | DESCRIPTION | SCANNERS (PARSERS) | RELATED OPTIONS | CUSTOM SCANNERS

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=makepp_scanning&sektion=1&manpath=FreeBSD+13.1-RELEASE+and+Ports>

home | help