Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
SYNTH(1)		FreeBSD	General	Commands Manual		      SYNTH(1)

NAME
     synth -- custom package repository	builder	for FreeBSD and	DragonFly

SYNOPSIS
     synth [help | configure | version | status	| purge-distfiles]
     synth [rebuild-repository | prepare-system	| upgrade-system]
     synth [everything | status-everything]
     synth [status | build | just-build	| install | force | test]
	   [file | port-origin ...]

DESCRIPTION
     The synth program is an advanced concurrent ports building	tool aimed at
     system administrators that	prefer or require the building of packages
     from source rather	than installing	official binary	packages.  synth will
     build packages in a clean environment can exactly mirror the system that
     they are built on,	it will	create local package respositories and install
     pkg(8) repository configuration file that causes locally built packages
     to	be used	with the highest priority, all while allowing the system to
     fully upgraded with a single command.

   synth version
     When synth	is executed with the version command, an unknown command, or
     no	command	at all,	it will	display	a short	block of text containing an
     identification, summary, version number, copyright, usage,	and a summary
     of	valid commands.

   synth help
     The help command displays lists each valid	command	with a short synopsis
     of	the command's function.	It also	elaborates the several commands	expect
     one or more port origins to be provided, or a path	to a file with the
     same information.

   synth configure
     The configure command launches an interactive menu	for providing the user
     with a clean method to configure synth.  It works on the concept of pro-
     files, and	the default profile ``LiveSystem'' is automatically installed
     with values that are determined by	scanning the system. Most of the con-
     figurable items are directories that are used to construct	the build
     environment, but they aren't limited to that.

     Ports directory	       If the PORTSDIR environment variable is set
			       when the	initial	profile	is created, it estab-
			       lishes the initial value. Otherwise, synth will
			       extract the value of PORTSDIR from
			       /usr/share/mk and use that.  If neither source
			       points to a valid path, /usr/dports and
			       /usr/ports will be successfully tried.  If
			       those don't exist either, synth will exit with
			       an error	message.

     Packages directory	       This is the location of the local repository
			       metadata	and all	the packages that are built by
			       synth.  It clearly must have lots of available
			       disk space. The default value is
			       /var/synth/live_packages	but it should be
			       altered if the system has a /var	partition with
			       limited space.

     Distfiles directory       This indicates where all	the ports source dis-
			       tribution files are stored, and the initial
			       value is	determined by scanning the existing
			       configuration. The defaults are defined by
			       ports and are /usr/ports/distfiles for
			       (FreeBSD) and /usr/distfiles for	(DragonFly).
			       Most users will want to keep the	initial	value
			       as that directory should	already	contain	dist-
			       files.

     Port options directory    This is the directory where all the selected
			       options for ports are cached.  The initial
			       value comes from	a system scan, so chances are
			       it has the correct value. However, if the user
			       would like a separate configuration area	for
			       port options, they would	create the new direc-
			       tory and	set this value accordingly (but	the
			       user would have to ensure the new location is
			       passed to the port when configuring port
			       options later in	that case).

     Build logs	directory      A log is	produced for every port	built (subse-
			       quent builds of the same	port will overwrite
			       previous	logs). This item dictates where	those
			       logs are	stored,	which is /var/log/synth	by
			       default.	If the /var partition is limited, the
			       user will definitely want to change this	loca-
			       tion as the uncompressed	logs can potentially
			       consume gigabyte	of space.

     Build base	directory      This is the mount point for all the builders.
			       When tmpfs is used, this	location requires very
			       little space as it contains empty directories
			       that serve as mount points. If tmpfs is not
			       used, then this location	should have access to
			       a lot available diskspace as it will provide
			       the work	area and virtual /usr/local for	each
			       builder.	The default location of
			       /usr/obj/synth-live should be fine for most
			       users (it generally has lots of space and tmpfs
			       is the default build mode).

     System root directory     Most users will use the base system as the
			       basis for the build environment,	and thus the
			       default setting of / is correct.	Advanced users
			       may use DESTDIR with installworld to create an
			       alternative build base (e.g. i386 cross-build
			       on amd64	architecture or	a modified base	that
			       serves as a fallout test	for an upcoming	fea-
			       ture). Those would be the users that would mod-
			       ify this	value.

     Compiler cache directory  If ccache is properly installed on the system,
			       set this	value to the location of the cache.
			       All the builders	will leverage it. It is	dis-
			       abled by	default.  It can be disabled again by
			       entering	any invalid path as a new value, e.g.
			       ``none''.

     Num. concurrent builders  The represents the number of ports that can be
			       simultaneously built on the system. The
			       selected	value is influenced by the number of
			       physical	cores and hyperthreads the system has,
			       the amount of memory on the system, the amount
			       of available swap, and if that swap is a	solid-
			       state drive or not. Generally memory is the
			       limiting	resource when tmpfs is used, so	the
			       default value for the number of builders	is
			       generally 75% of	the number of CPUs of the sys-
			       tem.  The user is free to balance jobs versus
			       builders	as well.

     Max. jobs per builder     If memory is constrained, it's often a better
			       performance tradeoff to reduce the number of
			       builders	and increase the number	of jobs	per
			       builder.	The default value varies depending on
			       the system, but it will never exceed 5.

     Use tmpfs for work	area   If the initial scan shows there is at least
			       1.25 Gb per builder, this item will default to
			       ``true''.  Building on tmpfs is a big speed
			       spot and	it's recommended if the	memory
			       resources allow it. This	item is	directed at
			       the work	area for building the port

     Use tmpfs for /usr/local  Before a	port can start building, all the build
			       and library dependencies	have to	be installed
			       into the	localbase. Extracting, installing and
			       later deinstalling is a lot of work for a disk,
			       so using	tmpfs here is another performance
			       boost. It is recommended	that users with	ade-
			       quate memory leave this enabled.

     Display using ncurses     During the concurrent building, the default
			       display is constructed using ncurses. It	shows
			       a lot of	good information and it	should be
			       used. The display will fall back	to a text mode
			       if the terminal doesn't support it.  This set-
			       ting forces text	mode unconditionally, but few
			       users will want or need to use this option.

     Fetch prebuilt packages   When active, this option	will scan the external
			       repository for suitable prebuilt	packages to
			       satisfy build requirements.  To be considered
			       suitable, the ABI, options, and dependencies of
			       the remote package must all match requirements.
			       If a package is located,	it will	be fetched and
			       placed in the packages directory	and later
			       incorporated into the local repository.	This
			       option is not active by default,	meaning	that
			       synth will normally build everything from
			       source and ignore external repositories.

     To	create alternative profiles, press the ``>'' key at the	prompt.	A new
     menu will appear that provides options to switch to another existing pro-
     file or create a new one that can be tailored and named. Any newly	cre-
     ated profile becomes the new default profile, but this can	be easily
     changed by	repeating the profile selection	process.

     Pressing the Alphabetic key associated with each configuration item
     brings up a prompt	to change the value. Directory entries must be valid
     paths except in the case of ccache. The boolean values are	modified with
     a single keypress (``T'' or ``F'')	and the	numeric	items require positive
     integers. After changing the values, the old menu returns and altered
     values are	marked with an asterisk.  Simply entering the ``Enter''	key
     (carriage return) saves the changes while pressing	the ``Escape'' key
     will discard them.

     If	more than one profile exists, an option	to delete profiles will	appear
     on	the menu.  By pressing the ``<'' key, a	new menu will appear that
     lists all the inactive profiles.  Entering	the number associated with one
     of	those profiles will immediately	delete the profile.  The deletion can-
     not be undone.

   synth upgrade-system
     The upgrade-system	command	automates the full upgrade of the system upon
     which synth is installed. It is expected that the ports tree has been
     updated to	the latest version before the command is executed. Using the
     ports tree	as a reference,	synth will query pkg(8)	to determine what is
     currently installed and which software has	newer versions available. It
     will rebuild those	packages and any package that depends libraries	they
     contain or	that have a runtime dependency on the rebuild packages.	In
     addition, any port	that has changed with respect to its options, depen-
     dencies, or ABI will be rebuilt as	well. After analysis, the concurrent
     builder will begin	building the set of packages that have been identified
     as	requiring a rebuild. When that phase is	complete, the local repository
     will be rebuilt without waiting for confirmation. Finally,	synth will
     command pkg(8) to upgrade the system using	the local repository, effec-
     tively bringing it	completely up to date.

   synth prepare-system
     The prepare-system	command	has the	same functionality of the
     upgrade-system command with the exception that the	packages will not be
     installed.	Once the local repository is fully rebuilt, the	command	exits.

   synth rebuild-repository
     The rebuild-repository command will perform a full	sanity check on	all
     packages present in the packages directory	and remove the ones which fail
     the check.	 The local Synth repository will be created (or	recreated)
     using the remaining packages.  This step is also done as part of several
     other commands, so	it mainly exists for scripting purposes.

   synth status
     The status	command	with no	arguments performs a dry-run of	the
     upgrade-system command. It	will not delete	obsolete packages, nor rebuild
     the local repository. It will list	all the	ports that will	be rebuild
     along with	a total, and it	also logs the same information to
     /tmp/synth_status_results.txt since the full list is often	longer than
     the terminal height.

   synth just-build [file | port-origin	...]
     The just-build command takes a list (one or more) of port origins (an
     origin is the combination of a category and the port directory, e.g.
     lang/python27) and	it will	builds the packages of those ports if neces-
     sary. If analysis determines the packages are already up to date, nothing
     is	done.

     This is the first of several commands that	take arguments.	The arguments
     are either	an unlimited number of port origins, e.g.

	    synth just-build editors/joe editors/nano editors/libreoffice

     or	they are limited to one, a path	to a file, e.g.

	    synth just-build /tmp/build.list

     A file that is equivalent to the first example would contain:

	    editors/joe
	    editors/nano
	    editors/libreoffice

     When the building task is complete, synth exits.

   synth build [file | port-origin ...]
     The build command differs from the	just-build command by asking the user
     if	they wish to rebuild the local repository when the building is com-
     plete. The	answer is usually ``N''	(for ``no'') because rebuilding	the
     repository	can take a few minutes to complete, and	it only	makes sense to
     do	when all the building is complete. Should the user opt to rebuild the
     repository, synth will then ask if	the user wishes	to install the pack-
     ages that come from the ports list	on the system.

   synth install [file | port-origin ...]
     The install command is similar to the build command except	that it	will
     not ask permission	to rebuild the repository or install the resulting
     packages. It will just do it all in sequence.

   synth force [file | port-origin ...]
     The force command is similar to the build command except that any pack-
     ages related to the ports list will be deleted first, regardless if they
     are up to date or not. This results in that every port on the given list
     will build.  As with the build command, synth will	request	permission to
     rebuild the repository and	possibly install the packages when the build-
     ing phase is complete.

   synth test [file | port-origin ...]
     The test command will pre-delete any existing packages related to the
     ports list	before rebuilding them with extra DEVELOPER_MODE tests and
     settings. When the	building is complete, it will just exit	without	offer-
     ing to rebuild the	repository.

   synth status	[file |	port-origin ...]
     This version of the status	command	provides a dry-run and reports on what
     would happen if the just-build command was	executed with the same argu-
     ments. The	output and logging is similar to the solo status command, but
     the build scope is	limited	to the given ports list.

   synth everything
     The everything command is not meant for most users. It will attempt to
     build every port in the ports tree, which can take	several	days depending
     on	the power and resources	of the building	machine. Even downloading the
     tens of gigabytes worth of	distfiles consumes significant time!

   synth status-everything
     The status-everything command is just a dry-run for the everything	com-
     mand. Since synth works through increment building	(that is, it will not
     build a port if an	up to date package already exists for it and all the
     packages dependencies are also in place) ,	the resultant build list can
     be	much shorter than the full list	of ports in the	tree. Only those rare
     users that	wish to	build the entire set of	ports would use	this command.

   synth purge-distfiles
     This is a useful command that will	first scan the entire ports tree to
     gather a complete list of distfiles used by the ports tree, and then it
     will scan the contents of the configured distfiles	directory. All source
     distribution files	that cannot be matched to at least one port in the
     tree will be removed.  This can result in gigabytes of recovered disk
     space if significant time has passed since	the distfiles directory	was
     last cleaned.

ENVIRONMENT
     PORTSDIR	   The PORTSDIR	environment variable is	only checked on	the
		   very	first signficant command of synth (meaning everything
		   except help and version) when the default profile is	cre-
		   ated. It is also checked when adding	new profiles. The
		   variable dictates the default location of the ports tree
		   with	a higher priority than the standard locations.

     SYNTHPROFILE  When	this environment variable is set to name of an exist-
		   ing synth profile, that profile will	be loaded when synth
		   is launch rather than the active profile.  It is a tempo-
		   rary	override; removing the variable	will allow the default
		   profile to load again.

     WHYFAIL	   When	this environment variable is defined to	any value, the
		   sanity check	routines will provide more detail on depen-
		   dency and option check failures on a	separate output	line.

     ENTERAFTER	   This	environment variable only applies with the test	com-
		   mand, and then only when a single port is specified.	 If
		   both	of those conditions are	satisfied, and the variable
		   has the value of:
		   - extract
		   - patch
		   - configure
		   - build
		   - stage
		   - install
		   - deinstall
		   then	an interactive build will be attempted.	First, all the
		   port's prerequisites	are built normally and synth will
		   return to text mode.	If everything built successfully, the
		   specified port will build up	to and including the point
		   specified by	ENTERAFTER. At that point, control will	be
		   given to the	user by	providing them with a tcsh shell with
		   the current directory set at	the root of the	builder.  When
		   the user is done, they should terminate the shell with the
		   exit	command	to signal synth	to clean up and	terminate.
		   This	is a developer tool to troubleshoot port build prob-
		   lems	and it is not intended for the average user.

     LOCK	   This	environment variable only applies with the test	com-
		   mand. When it is defined to any value, the builder's	local-
		   base	is remounted read-only prior to	the configure phase
		   and restored	to read/write permissions after	building is
		   complete.  This is a	diagnostic tool	aimed at pinpointing
		   file	system violations as a port is not supposed to write
		   to localbase	prior to staging.  Note	that the localbase
		   tmpfs setting will considered as False in this mode,	so
		   building may	be noticeably slower than normal.

     SELFTEST	   This	environment variable only applies with the test	com-
		   mand.  When it is defined, the ``test'' target is executed
		   as part of the build	cycle; this target is not executed
		   under any other circumstances.

FILES
     The following files may be	used or	created:

     synth.ini	       This is the configuration file. It is automatically
		       generated and manipulated using the interactive config-
		       uration menu. There is no need to touch this file other
		       than to remove unwanted profiles	as currently that is
		       not possible from the program itself. It	is normally
		       located at /usr/local/etc/synth/synth.ini

     <profile>-make.conf
		       This is an optional, user-provided file.	If it exists,
		       the builder's /etc/make.conf will be appended with the
		       contents	of this	file. For the default profile, the
		       file would normally be located at
		       /usr/local/etc/synth/LiveSystem-make.conf

     <profile>-environment
		       This is an optional, user-provided file.	It contains a
		       list of name-value pairs	joins with an equals sign
		       (e.g. HTTP_PROXY=http://proxyserver:8888), one pair per
		       line.  If the file exists, these	variables will be
		       defined in the builder environment.  For	the default
		       profile,	the file would normally	be located at
		       /usr/local/etc/synth/LiveSystem-environment

     <profile>-private.key
		       This is one of two files	required to support the	sign-
		       ing of the repository by	a local	RSA key.  It is	the
		       output of the openssl genrsa command; see the EXAMPLE
		       section for how to generate it.	The file permissions
		       are required to be set at 400 as	well.  For the default
		       profile,	the file is normally located at
		       /usr/local/etc/synth/LiveSystem-private.key

     <profile>-public.key
		       This is the second of two files required	to support the
		       signing of the repository by a local RSA	key.  synth
		       must confirm the	public counterpart of the signing key
		       is available because its	path is	required for the gen-
		       erated pkg repository configuration file.  For the
		       default profile,	the file is normally located at
		       /usr/local/etc/synth/LiveSystem-public.key

     <profile>-signing_command
		       This is one of two files	required to support the	sign-
		       ing of the repository by	an external signing server.
		       It consists of a	single line containing the command;
		       see the EXAMPLE section.	 For the default profile, the
		       file is normally	located	at
		       /usr/local/etc/synth/LiveSystem-signing_command

     <profile>-fingerprint
		       This is the second of two files required	to support the
		       signing of the repository by an external	signing
		       server.	It consists of a single	line containing	the
		       SHA 256 hash of the public key that is the counterpart
		       of the key that signed the repository.  For the default
		       profile,	the file is normally located at
		       /usr/local/etc/synth/LiveSystem-fingerprint

     synth_status_results.txt
		       Whenever	a status command is run, a list	of ports that
		       would build with	the actual command is produced at
		       /tmp/synth_status_results.txt

     synth_prefetch_list.txt
		       Whenever	a status command is run, and the option	to use
		       suitable	prebuilt packages has been selected, a list of
		       packages	that will be fetched as	a result will be writ-
		       ten to /tmp/synth_prefetch_list.txt

     index.html	       For every build run, Synth installs a web summary at
		       the Report subdirectory of the build logs directory.
		       For the default profile,	the file would be located at
		       /var/synth/logs/Report/index.html.  To remotely view
		       the web report with a browser, a	web server must	be
		       present and configured to expose	the logs directory.
		       The web report is dynamic and updated 10	times per
		       minute.	The complete history is	available and search-
		       able, with quick-filters	on "Built", "Failed",
		       "Ignored" and "Skipped" fields as well as on the	"No."
		       column which is useful to examine cascaded skips.  The
		       search filter can be quickly cleared by clicking	on the
		       "Total" field.

     port logs	       Every single attempt at building	a port results in a
		       build log. The name of the log is always	in the form of
		       [category]___[port].log so that subsequent builds will
		       overwrite previous attempts. They are created in	the
		       directory specified by the configuration.

     hook_run_start    If this file exists, and	it is executable (or it	is a
		       symbolic	link that leads	to an executable file) then it
		       will be executed	by synth at the	start of the build.
		       The same	requirement exists for all hooks. This hook,
		       and the next five hooks all define the following	common
		       environment variables: PROFILE, DIR_PACKAGES,
		       DIR_REPOSITORY, DIR_PORTS, DIR_OPTIONS, DIR_DISTFILES,
		       DIR_LOGS, DIR_BUILDBASE.	 Uniquely, this	hook also
		       defines PORTS_QUEUED which is the number	of ports that
		       synth will attempt to build. This file must be colated
		       with synth.ini, so that would be
		       /usr/local/etc/synth/hook_run_start for most installa-
		       tions.

     hook_run_end      This hook is fired when the build completes. In addi-
		       tion to the common definitions, it defines PORTS_BUILT,
		       PORTS_FAILED, PORTS_IGNORED, and	PORTS_SKIPPED in the
		       environment, which represents the results of the	build.

     hook_pkg_success  This hook is fired whenever a builder successfully
		       packages	a port.	In addition to the common definitions,
		       it defines RESULT=success, ORIGIN, and PKGNAME in the
		       environment.

     hook_pkg_failure  This hook is fired whenever a builder fails to package
		       a port. In addition to the common definitions, it
		       defines RESULT=failure, ORIGIN, and PKGNAME in the
		       environment.

     hook_pkg_ignored  This hook is fired prior	to the start of	a build	when
		       synth prescans the queue	for ports that are to be
		       ignored according to the	ports tree. Once the build has
		       started,	this hook is never fired again.	In addition to
		       the common definitions, it defines RESULT=ignored,
		       ORIGIN, and PKGNAME in the environment.

     hook_pkg_skipped  This hook is fired whenever a package gets ignored or
		       fails which in turns cascades into a number of ports
		       getting removed from the	build queue due	to the depen-
		       dency's failure.	This may fire before the build starts
		       due to ignored prescanning, or any time during the
		       build. In addition to the common	definitions, it
		       defines RESULT=skipped, ORIGIN, and PKGNAME in the
		       environment.

SEE ALSO
     pkg(8), pkg-rep(8)

NOTES
   Graceful exit
     During development, the interrupt signal (SIGINT) was captured and	han-
     dled. It turns out	this handling extended to the builders,	so any event
     resulting in a SIGINT in the build	would lock up the builder permanently.
     There's no	apparent workaround for	this given the design, so pressing
     Control-C is no longer handled. It	will stop the build, but it will leave
     everything	in bad state. The next significant call	to synth will attempt
     to	clean that up though. If a user	wants to stop the build, the best
     approach is to type the Control-Q combination. The	program	will shut down
     as	soon as	it can.	During the building phase no new builds	will start
     after the shutdown	begins,	but the	active builders	will continue until
     completion.

   Unprivileged	users
     At	this time, synth can only be run by the	root user.

   Impulse indicator
     A common question is "What	does the Impulse indicator on the display
     mean?"  It	is the package build rate (packages/hour) over the last	500
     seconds.  For the first roughly ten minutes of a bulk run,	the
     ``Impulse'' rate and the ``Pkg/Hour'' rate	will be	identical. After that,
     the ``Impulse'' build rate	will differ from the rate averaged over	the
     entire elapsed time, often	by a great amount.  It's just a	way to gauge
     the package production rate over the previous few minutes.

   Supported architectures
     Synth is written in Ada, and thus an Ada compiler or cross-compiler is
     required to build it. For FreeBSD,	that means synth is currently limited
     to	amd64 and i386.	It is feasible to port GNAT to the ARMv7 architecture
     as	this has already been done (See	lang/gnatdroid-armv7 port) but it's
     not a trivial task	to do it. Other	architectures would be challenging,
     although not impossible, to support.

EXAMPLE
     To	activate the signing of	default	profile's repository with a local RSA
     key, generate the two key files:

	   % cd	/usr/local/etc/synth
	   % openssl genrsa -out LiveSystem-private.key	2048
	   % chmod 0400	LiveSystem-private.key
	   % openssl rsa -in LiveSystem-private.key -out \
	     LiveSystem-public.key -pubout

     To	activate the signing of	default	profile's repository with an external
     signing server, two files need to be created.  It's assumed the signing
     server has	already	been configured	as described by	the pkg-rep(8) man
     page, and that the	keys are located in the	repo.pub and repo.key files.
     On	the signing server which is 192.168.0.44 in this example:

	   % sha256 -q repo.pub	> LiveSystem-fingerprint
	   % echo "/usr/bin/ssh	192.168.0.44 /usr/local/bin/sign.sh" \
	     > LiveSystem-signing_command

     Then move both files to /usr/local/etc/synth directory on the build
     server.  The build	server root user needs to have a passphrase-free SSH
     key access	to the signing server in order for Synth to sign the reposi-
     tory.

HISTORY
     synth was conceived long before coding began in December of 2015. It was
     designed to be system-agnostic from the beginning,	and in particular sup-
     port for FreeBSD and DragonFly was	added simulaneously. It	was hoped that
     it	would attract users of the older PortMaster and	PortUpgrade ports man-
     agement tools by providing	them with a superior and well-maintained
     alternative.

     synth was written entirely	by John	Marino <marino@FreeBSD.org> and
     released under the	Internet Software Consortium license.

FreeBSD	Ports 11.2	       December	6, 2016		    FreeBSD Ports 11.2

NAME | SYNOPSIS | DESCRIPTION | ENVIRONMENT | FILES | SEE ALSO | NOTES | EXAMPLE | HISTORY

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=synth&sektion=1&manpath=FreeBSD+12.0-RELEASE+and+Ports>

home | help