Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages


home | help

       makepp_signatures -- How	makepp knows when files	have changed

       C: C,
	 c_compilation_md5,  M:	"md5",	P: "plain",  S:	"shared_object",
       X: "xml",

       Each file is associated with a signature, which is a string that
       changes if the file has changed.	 Makepp	compares signatures to see
       whether it needs	to rebuild anything.  The default signature for	files
       is a concatenation of the file's	modification time and its size,	unless
       you're executing	a C/C++	compilation command, in	which case the default
       signature is a cryptographic checksum on	the file's contents, ignoring
       comments	and whitespace.	 If you	want, you can switch to	a different
       method, or you can define your own signature functions.

       How the signature is actually used is controlled	by the build check
       method (see makepp_build_check).	 Normally, if a	file's signature
       changes,	the file itself	is considered to have changed, and makepp
       forces a	rebuild.

       If makepp is building a file, and you don't think it should be, you
       might want to check the build log (see makepplog).  Makepp writes an
       explanation of what it thought each file	depended on, and why it	chose
       to rebuild.

       There are several signature methods included in makepp.	Makepp usually
       picks the most appropriate standard one automatically.  However,	you
       can change the signature	method for an individual rule by using
       ":signature" modifier on	the rule which depends on the files you	want
       to check, or for	all rules in a makefile	by using the "signature"
       statement, or for all makefiles at once using the "-m" or
       "--signature-method" command line option.

   Mpp::Signature methods included in the distribution
       plain (actually nameless)
	   The plain signature method is the file's modification time and the
	   file's size,	concatenated.  These values are	quickly	obtainable
	   from	the operating system and almost	always change when the file

	   Makepp used to look only at the file's modification time, but if
	   you run makepp several times	within a second	(e.g., in a script
	   that's building several small things), sometimes modification times
	   won't change.  Then,	hopefully the file's size will change.

	   If the case where you may run makepp	several	times a	second is a
	   problem for you, you	may find that using the	"md5" method is
	   somewhat more reliable.  If makepp builds a file, it	flushes	its
	   cached MD5 signatures even if the file's date hasn't	changed.

	   For efficiency's sake, makepp won't reread the file and recompute
	   the complex signatures below	if this	plain signature	hasn't changed
	   since the last time it computed it.	This can theoretically cause a
	   problem, since it's possible	to change the file's contents without
	   changing its	date and size.	In practice, this is quite hard	to do
	   so it's not a serious danger.  In the future, as more filesystems
	   switch to timestamps	of under a second, hopefully Perl will give us
	   access to this info,	making this failsafe.

	   This	is the method for input	files to C like	compilers, if your
	   Perl	supports MD5 (only optionally available	up to 5.6.2).  It
	   checks if a file's name looks like C	or C++ source code, including
	   things like Corba IDL.  If it does, this method applies.  If	it
	   doesn't, it falls back to plain signatures for binary files
	   (determined by name or else by content) and else to "md5".  Alas
	   you can currently tune what is to be	considered C or	binary only by
	   subclassing "Mpp::Signature::c_compilation_md5" to get a new
	   signature type and there overriding or extending the	methods
	   "recognizes_file" and "excludes_file".

	   The idea is to be independent of formatting changes.	 This is done
	   by pulling everything up as far as possible,	and by eliminating
	   insignificant spaces.  Words	are exempt from	pulling	up, since they
	   might be macros containing "__LINE__", so they remain on the	line
	   where they were.

	       // ignored comment

	       #ifdef XYZ
		   #include <xyz.h>

	       int a = 1;

	       void f
		   int b
		   a +=	b + ++c;

		   /* more ignored comment */

	   is treated as though	it were

	       #ifdef XYZ

	       int a=1;

	       void f(

	       int b){

	       a+=b+ ++c;}

	   That	way you	can reindent your code or add or change	comments
	   without triggering a	rebuild, so long as you	don't change the line
	   numbers.  (This signature method recompiles if line numbers have
	   changed because that	causes calls to	"__LINE__" and most debugging
	   information to change.)  It also ignores whitespace and comments
	   after the last token.  This is useful for preventing	a useless
	   rebuild if your VC adds lines at a "$""Log$"	tag when checking in.

	   This	method is particularly useful for the following	situations:

	   o   You want	to make	changes	to the comments	in a commonly included
	       header file, or you want	to reformat or reindent	part of	it.
	       For one project that I worked on	a long time ago, we were very
	       unwilling to correct inaccurate comments	in a common header
	       file, even when they were seriously misleading, because doing
	       so would	trigger	several	hours of rebuilds.  With this
	       signature method, this is no longer a problem.

	   o   You like	to save	your files often, and your editor (unlike
	       emacs) will happily write a new copy out	even if	nothing	has

	   o   You have	C/C++ source files which are generated automatically
	       by other	build commands (e.g., yacc or some other
	       preprocessor).  For one system I	work with, we have a
	       preprocessor which (like	yacc) produces two output files, a
	       ".cxx" and a ".h" file:

		   %.h %.cxx: %.qtdlg $(HLIB)/Qt/qt_dialog_generator
		       $(HLIB)/Qt/qt_dialog_generator $(input)

	       Every time the input file changed, the resulting	.h file	also
	       was rewritten, and ordinarily this would	trigger	a rebuild of
	       everything that included	it.  However, most of the time the
	       contents	of the .h file didn't actually change (except for a
	       comment about the build time written by the preprocessor), so a
	       recompilation was not actually necessary.

       md5 This	is the default method, if your Perl supports MD5 (optionally
	   available up	to 5.6.2).  Computes an	MD5 checksum of	the file's
	   contents, rather than looking at the	file's date or size.  This
	   means that if you change the	date on	the file but don't change its
	   contents, makepp won't try to rebuild anything that depends on it.

	   This	is particularly	useful if you have some	file which is often
	   regenerated during the build	process	that other files depend	on,
	   but which usually doesn't actually change.  If you use the "md5"
	   signature checking method, makepp will realize that the file's
	   contents haven't changed even if the	file's date has	changed.  (Of
	   course, this	won't help if the files	have a timestamp written
	   inside of them, as archive files do for example.)

	   This	method only works if you have the utility "nm" in your path,
	   and it accepts the "-P" option to output Posix format.  In that
	   case	only the names and types of symbols in dynamically loaded
	   libraries become part of their signature.  The result is that you
	   can change the coding of functions without having to	relink the
	   programs that use them.

	   In the following command the	parser will detect an implicit
	   dependency on $(LIBDIR)/,	and build it if	necessary.
	   However the link command will only be reperformed whenever the
	   library exports a different set of symbols:

	       myprog: $(OBJECTS) :signature shared_object
		   $(LD) -L$(LIBDIR) -lmylib $(inputs) -o $(output)

	   This	works as long as the functions'	interfaces don't change.  But
	   in that case	you'd change the declaration, so you'd also need to
	   change the callers.

	   Note	that this method only applies to files whose name looks	like a
	   shared library.  For	all other files	it falls back to
	   "c_compilation_md5",	which may in turn fall back to others.

	   These are two similar methods which treat xml canonically and
	   differ only in their	handling of whitespace.	 The first completely
	   ignores it around tags and considers	it like	a single space
	   elsewhere, making the signature immune to formatting	changes.  The
	   second respects any whitespace in the xml, which is necessary even
	   if just a small part	requires that, like a "<pre>" section in an
	   xhtml document.

	   Common to both methods is that they sign the	essence	of each	xml
	   document.  Presence or not of a BOM or "<?xml?>" header is ignored.
	   Comments are	ignored, as is whether text is protected as "CDATA" or
	   with	entities.  Order and quoting style of attributes doesn't
	   matter, nor does how	you render empty tags.

	   For any file	which is not valid xml,	or if the Expat	based
	   "XML::Parser" or the	"XML::LibXML" parser is	not installed, this
	   falls back to method	md5.  If you switch your Perl installation
	   from	one of the parsers to the others, makepp will think the	files
	   are different as soon as their timestamp changes.  This is because
	   the result of either	parser is logically equivalent,	but they
	   produce different signatures.  In the unlikely case that this is a
	   problem, you	can force use of only "XML::LibXML" by setting in

	       $Mpp::Signature::xml::libxml = 1;

   Extending applicability
       The "C" or "c_compilation_md5" method has a built in list of suffixes
       it recognizes as	being C	or C-like.  If it gets applied to other	files
       it falls	back to	simpler	signature methods.  But	many file types	are
       syntactically close enough to C++ for this method to be useful.	Close
       enough means C++	comment	and string syntax and whitespace is
       meaningless except one space between words (and C++'s problem cases "-
       -", "+ +", "/ *"	and "< <").

       It (and its subclasses) can now easily be extended to other suffixes.
       Anyplace	you can	specify	a signature you	can now	tack on	one one	of
       these syntaxes to make the method accept	additional filenames:

	   One or more comma-separated suffixes	can be added to	the method by
	   a colon.  For example "C.ipp,tpp" means that	besides	the built in
	   suffixes it will also apply to files	ending in .ipp or .tpp,	which
	   you might be	using for the inline and template part of C++ headers.

	   This	is like	the previous, but instead of enumerating suffixes, you
	   give	a Perl regular expression to match the ones you	want.  The
	   previous example would be "C.(ipp|tpp)" or "C.([it]pp)" in this

	   Without a dot the Perl regular expression can match anywhere	in the
	   file	name.  If it includes a	slash, it will be tried	against	the
	   fully qualified filename.  So if you	have C++ style suffixless
	   headers in a	directory include, use "C(/include/)" as your
	   signature method.  However the above	suffix example would be	quite
	   nasty this way, "C(\.(?:ipp|tpp)$$)"	or "C(\.[it]pp$$)" because "$"
	   is the expansion character in makefiles.

       Signature methods apply to all files of a rule.	Now if you have	a
       compiler	that takes a C like source code	and an XML configuration file
       you'd either need a combined signature method that smartly handles both
       file types, or you must choose an existing method which will not	know
       whether a change	in the other file is significant.

       In the future signature method configuration may	be changed to
       filename-pattern, optionally per	command.

   Custom methods
       You can,	if you want, define your own methods for calculating file
       signatures and comparing	them.  You will	need to	write a	Perl module to
       do this.	 Have a	look at	the comments in	"Mpp/" in the
       distribution, and also at the existing signature	algorithms in
       "Mpp/Signature/*.pm" for	details.

       Here are	some cases where you might want	a custom signature method:

       o   When	you want all changes in	a file to be ignored.  Say you always
	   want	dateStamp.o to be a dependency (to force a rebuild), but you
	   don't want to rebuild if only dateStamp.o has changed.  You could
	   define a signature method that inherits from	"c_compilation_md5"
	   that	recognizes the dateStamp.o file	by its name, and always
	   returns a constant value for	that file.

       o   When	you want to ignore part	of a file.  Suppose that you have a
	   program that	generates a file that has a date stamp in it, but you
	   don't want to recompile if only the date stamp has changed.	Just
	   define a signature method similar to	"c_compilation_md5" that
	   understands your file format	and skips the parts you	don't want to
	   take	into account.

perl v5.32.1			  2012-02-07		  MAKEPP_SIGNATURES(1)


Want to link to this manual page? Use this URL:

home | help