Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages


home | help
DPHYS-CONFIG(1)		    General Commands Manual	       DPHYS-CONFIG(1)

       dphys-config - daily auto-install/update	and/or remove config files

       dphys-config [-f	filter]	[-cqvD]

       dphys-config -h

       dphys-config  installs/updates  and/or  removes	config	files. It also
       triggers	commands after an new/updated config file is available or  be-
       fore  an	 existing  config  file	will disappear.	It can be run by hand,
       from cron and/or	from init.d.

       Get an list of config files from	an configuration server. For each file
       in  the	list retrieve that file	from the same server, and only install
       it if it	is new or changed relative to what is already here. If a  file
       is  newly  installed (or	changed) then run an postinstall script, which
       may trigger actions which are wanted to process the new config (such as
       inserting  data	from an	config file into an database). Also remove un-
       wanted files. If	doing so first run an  preremove  script  to  tidy  up

       This  is	part of	the D-PHYS (ETH	Zuerich, Departement of	Physics) auto-
       matic system operation and maintenance setup.

       -c     configname: Use this set of config  files	 instead  of  hostname
	      set. Useful for chroot or	vhost installs,	or for tests.

       -f  filter
	      filter: Only process lines which match the filter	spec.

       -q     quiet: Don't produce an running report of	activities.

       -v     verbose: Give large volume output, where sensible.

       -D     Debug: Activate an debug option. See source for how to use this.

       -h     help: Output help	text, and then abort operation.

       The config files	/etc/dphys-config (sitewide) and ~/.dphys-config (per-
       sonal) allow the	admin and users	to set up the working environment  for

       These  config  files are	sh script fragments full of assignments, which
       are sourced, in above row, later	config	files  assignments  overriding
       earlier ones.  Standard sh syntax rules apply. Assignments are:

	      Sets the base directory in which all temporary files are stored.
	      It defaults to /var/tmp (for enough size	and  safe  operation).
	      Some  users may like to use /tmp for higher speed	(tmpfs)	or au-
	      tomatic deletion at boot time.  Standard sh syntax rules	apply.
	      Assignments are:

	      Sets  the	base URL to which all */_hostname_/_filename_ combina-
	      tions are	added when wget-ing config files. This can be an http:
	      or ftp: or whatever other	type of	URL which wget understands and
	      can fetch	an file	from. Additionally it can be  an  file:	 (this
	      may  be  from an NFS server) URL,	in which case wget is bypassed
	      and the files fetched directly using cp. It defaults to the  er-
	      ror   message   generating   and	aborting  invalid  setting  of
	      tory,  as	 there	is  no sensible	default	possible. You must set
	      this to where ever your config files should be taken from.

	      Selects the name for which set of	configuration files  shall  be
	      used for this host. Defaults to `hostname`.

	      Sets  an	regexp	which selects which lines from the config file
	      list are processed. Defaults to .* (all).

	      Log to syslog that dphys-config has run. Good  to	 see  if  cron
	      and/or init.d have done their job. Defaults to yes.

       The  config  file list dphys-config.list, which is found	via above set-
       tings, and is downloaded	 to  /etc/dphys-config.list  or	 ~/.dphys-con-
       fig.list,  then	allows	the  admin to list what	config files are to be
       fetched and installed/updated or	removed, and what scripts to  run  for
       them.  These can	be each	given for the entire site (= all hosts)	and/or
       group and/or each host, or even	merged	from  site+group+host  subsec-

	      site admin config

	      users personal config

	      roots config file	list gets stored here

	      users config file	list gets stored here

	      site-global (all hosts) common (usually, or group-global or host
	      specific)	config file list

	      facultative  host-specific  (usually)  or	  group-specific   in-
	      clude-able  subsection(s)	 to be added to	above config file list
	      dphys-config.list. We often use *.group (one per group of	users)
	      and  *.host  (per	 host),	sometimes also *.base (all host	types)
	      and   *.workstation   (only   workstation	  hosts)   subsections
	      $CONF_BASEURL/`hostname`/<file-name>  actual  config  files  re-
	      ferred to	in config file list, common section (usually the  only
	      section)	 $CONF_BASEURL/`hostname`/<file-name>.*	   facultative
	      host-specific (usually) or group-specific	 include-able  subsec-
	      tion(s) to be added to above config file <file-name>

       The  config file	list to	be used	for checking what files	need to	be in-
       stalled/updated or removed and its subsections  included	 by  #@include
       lines are merged	to one list file, analog to cpp	#include.

       These are all fetched via wget (or cp for file:), adding	their names to
       the  user-defined  base	URL  in	 CONF_BASEURL,	and  then  merged.  So
       CONF_BASEURL  can  be  any  URL	that wget understands http: or ftp: or
       whatever	else, or file:.

       The format of the resulting concatenated	file must  consist  of	lines,
       one per config file, of following format:


       Where the 3 fields have following meanings:

		     Name  of the config file to be installed/updated. Must be
		     only the base part	of the filename	on the server, without
		     URL and hostname before it, and without any .* subsection
		     endings after it, as these	are  all  auto-added  whenever
		     they  are	needed.	If this	is set to - the	line specifies
		     an	config file to be removed

		     Full directory or full filename  (directory+filename)  of
		     where  the	 file is to be placed on the target system. If
		     only an directory is given	(any name  that	 ends  in  /),
		     then  the	above  file-name (inclusive any	directories in
		     it) will be automatically added to	it. For	removing  this
		     must  be the full filename	(or an directory name (without
		     an	/) if an entire	directory and its contents shall  dis-
		     apper). An	directory name ending with / is	not processed,
		     to	prevent	incomplete edits (filename replaced by -,  but
		     not  added	 to directory) from killing entire directories
		     (such as say all of /etc/ :-))

		     Full command (directory+filename, with parameters)	of  an
		     command  to be run, after this config file	has been newly
		     installed or changed/updated, or before this config  file
		     is	 removed. This can also	be multiple commands separated
		     by	; separators. Useful for doing	chown/chmod  to	 files
		     that  need	 it.  If the marker {} appears in the command,
		     this will be substituted by the filename the config  file
		     is	going to be installed as. This is analog to find -exec
		     filename substitution

       Lines which begin with an # are regarded	as comments,  and  don't  have
       any  effect  anything  (Lines extended with one are chopped off at that
       point). The same	applies	for empty lines.

       If the first line of the	config file list, or any config	 file  fetched
       on  its	behalf,	 has the special format	#@dphys-config-preprocess [ac-
       tion...]	then this line will be stripped, and the rest of the file will
       be preprocessed.	Depending on the list of actions present and their or-
       der (repeats are	allowed) the file will be procesed. Valid actions are:

		     Anything inside backticks (``) will be executed as	a com-
		     mand,  and	its stdout will	then be	substituted for	the ``
		     expression. This is analog	to sh backtick substitution

	      if     For any line beginning with #@if the  stuff  between  the
		     #@if  and	the first ; character will be executed as com-
		     mand, and if it returns true, everything after the	; will
		     be	 left,	else  the entire line will be removed. This is
		     analog to shell if	... ; conditional execution

		     For any line beginning with #@include  the	 rest  of  the
		     line  is  regarded	 as  an	subsection name, which will be
		     added to the base filename, and then the  resulting  sub-
		     section file fetched (also	by wget	or cp) and substituted
		     for the line. This	is analog to an	 C  preprocessor  #in-
		     clude oder	an shell .  include

       The  following  allows  you to fetch all	your config file lists from an
       HTTP VirtualHost	called  under  its  subdirectory

       In  file	 /etc/dphys-config,  on	 every host, so	it can find the	config
       file server:

       # system	will use ${CONF_BASEURL}/`hostname`/<file-name>*

       We advise using an subdirectory	here,  because	other  /* directories may already contain other admin stuff you
       put on the same VirtualHost. Such as software packages, site news, etc.

       For dphys-config	to be useful you then need to make config  file	 lists
       for  it.	  And  provide	the actual config files	that can be installed,
       driven by the lists.  This is the largest job, as it basically  amounts
       to  extracting all your relevant	config work from your site. Also known
       as reengineering	your site.

       Assuming	your VirtualHost on has as its Document-
       Root  /vhost/www.admin, you would then begin with an pseudo-host	Direc-
       tory for	site-global common stuff: /vhost/www.admin/dphys-config/SITE/.

       If your hosts are organised in groups with group-global common  configs
       (such  as  professors,  students,  staff), make an pseudo-host for each
       group, such as:	/vhost/www.admin/dphys-config/PROFS/  and  */STUDENTS/
       and */STAFF/.

       Then for	host specific stuff, assuming systems called
       to, stud1 to stud20, staff1 to	 staff5,  server1  and
       server2,	 make  for each	its own	directory: /vhost/www.admin/dphys-con-
       fig/prof1/ (and so on).

       Note that we suggest using CAPITALS for pseudo-hosts and	lowercase  for
       actual hosts. This avoids name space collisions.	You can	also use loops
       like for	host in	[a-z]* ; do ...	; done to work	(say  generating  sym-
       links  to  an  new  config file in all hosts). Well at least you	can do
       this so long no one goes	and sets LANGUAGE= or similar junk, then  bash
       (or  libc?)  will  hapily  screw	up case	sensitivity and	produce	random
       lossage (yes, it	was painfull).

       After this add to /vhost/www.admin/dphys-config/SITE/, the actual  con-
       fig  files  as  far  as they are	not host specific, or at least have an
       common section to all hosts. Example this would be /etc/hosts for  all,
       an  common section for /etc/motd, common	or all for,	common
       for inetd.conf, nothing for the ssh hostkeys.

       Then add, to  an	 group,	 say  /vhost/www.admin/dphys-config/STUDENTS/,
       whatever	 is specific to	that group. Example this may be	an entire spe-
       cial motd for the many changing users, or just an  to	 #@in-
       clude into the common one.

       Then  for each host in its /vhost/www.admin/dphys-config/prof1/ (or so)
       add all that is specific	to it. Such as its ssh key files. And its  own,  it it needs one. Same its if	it is going to
       offer special stuff. An configs for services only this host has such as

       Then  for  each host add	symlinks to the	SITE or	group versions that it
       is  to  use  for	 common	 stuff,	 like  on  /vhost/www.admin/dphys-con-

	.../dphys-config/stud1/dphys-config.list -> ../SITE/dphys-config.list
	.../dphys-config/stud1/hosts ->	../SITE/hosts
	.../dphys-config/stud1/inetd.conf -> ../SITE/inetd.conf
	.../dphys-config/stud1/motd -> ../SITE/motd
	.../dphys-config/stud1/ -> GROUP/motd
	.../dphys-config/stud1/GROUP ->	../STUDENTS
	.../dphys-config/stud1/ -> ../SITE/

       In   the	  /vhost/www.admin/dphys-config/SITE/	directory   place  the
       site-global common dphys-config.list for	 all  your  hosts,  containing
       stuff like this:

       # SITE dphys-config.list	- just example stuff, for our exemplaric site
       # basics
       hosts:/etc/			# simply works,	no command
       motd:/etc/			# this will be assembled group specific
       inetd.conf:/etc/:/etc/init.d/inetd restart  # needs an command to reload restart  # not in /etc
       # ssh restart only after	last file, and ensure file modes for each file
       ssh_host_key:/etc/ssh/:chown root:root {}; chmod	600 {}
       ssh_host_rsa_key:/etc/ssh/:chown	root:root {}; chmod 600	{}
       ssh_host_dsa_key:/etc/ssh/:chown	root:root {}; chmod 600	{}; /etc/init.d/sshd restart
       # load stuff into an existing database file
       seed.debconf:/etc/:debconf-set-selections {}
       # other stuff
       daemon1-conf:/etc/daemon1/conf	# rename so names can differ on	server
       daemon1/conf:/etc/		# same as above, but with directories on server
       testing:/etc/			# put something	in there for an	test
       # delete	some stuff
       -:/etc/testing			# change to above test to get rid of it	again
       -:/etc/				# you will get a warning if you	leave this
       #-:/etc				# you would reinstall your system after	the resulting  rm -rf /etc  :-)
       # and some errors
       #only-an-name			# you would get	an error: no place on target
       #only-an-name:			# you would get	an error: no place on target
       #:only-an-place			# you would get	an error: no file to install

       For  special  services  add an on	each host that
       has  special  config  files  not	 present  on  others,	such   as   on

       # server2	- only used on our web server
       httpd.conf:/etc/apache/httpd.conf:/etc/init.d/apache restart

       You  can	also use dphys-config to run arbitrary commands, whenever con-
       fig files are installed/updated or removed, to modify  existing	config
       files,  or more likely modify complex config databases which can	not be
       provided	as files, but where one	can provide edit info as files.

       dphys-config can	even install scripts to	use as above commands (or even
       just to run scripts while installing), such as into /usr/local/sbin/.

       For  this  make	an ../SITE/local/sbin/ directory, place	the scripts in
       there (such as ../SITE/local/sbin/dphys-config-_whatever_), and symlink
       local  to ../SITE/local on each host, and then add config lines for the
       scripts,	with the command to trigger them, giving something like	this:

       local/sbin/dphys-config-<whatever>:/usr/:chmod 755 {}; {}  # chmod and run

       It this script processes	an config file your will want it to be run  if
       either  the  script or the config file is updated, so add the script to
       the laters line as well:

       dphys-config-<whatever>:/etc/:if	[ -x /usr/local/sbin/dphys-config-<whatever> ] ; then /usr/local/sbin/dphys-config-<whatever> ;	fi  # run also here

       Finally,	new hosts can then later simply	be added, by  making  the  new
       hosts  directory	 and  copying  all files and symlinks from an existing
       host of the same	group. Such as by doing:

       mkdir student21
       tar -cf - -C student1 . | tar xpf - -C student21

       To then run dphys-config	by hand	(say for tests), type on the host:


       But usually you will want  to  run  dphys-config	 automatically,	 every
       night  (or  if  a  machine was/is switched off, at every	boot), to keep
       your configs up to date.

       For nightly updates the best thing is to	use an cron job	on every host.
       03:00  to  03:59	 is  most likely idle time. Use	an line	like this one,
       with the	cron option to avoid an	load peak on the config	 file  server,
       by random delaying the run by 0..3599 seconds, and with stdout and str-
       err thrown away to avoid	getting	an mail	from every host, as error mes-
       sages are also allways sent to syslog:

       0 3 * * * root dphys-config cron	> /dev/null 2>&1

       To  catch  machines  switched off over night, with no cron run on them,
       also run	an init.d script. Use an script	like this one, also with  std-
       out  and	 stderr	thrown away to avoid cluttering	your boot console out-

       # /etc/init.d/dphys-config - boot time automatic	config updates,	if no cron
       case "$1" in
	   dphys-config	init > /dev/null 2>&1
       exit 0

       If dphys-config is to be	used to	distribute all config files, this will
       also  include  files  which  are	security relevant, such	as ssh private
       keys (host key or  (root)  user	authentification),  SSL	 certificates,
       passwd and shadow, lilo.conf, software license keys, etc.

       As  all	files are most likely fetched from an http: URL, measures must
       be taken	to secure the config website  from  other  people  downloading
       them.  We  here	use  an	restriction to only IP addresses registered as
       hosts in	our NIS	server,	and additionally run  identd  on  all  allowed
       hosts,  and  require the	wget process opening the HTTP connection to be
       running by user root, and so also require dphys-config to run as	root.

       To avoid	sniffing it is recommended to give wget	an https: URL.

       Config files are	read by	wget from an webserver,	 so  they  lose	 their
       owners and modes. So the	commands triggered on their lines must be used
       to chown/chmod them to proper values.

       When used together with dphys-admin, dphys-config should	run  as	 first
       (earlier	 cron  and  init.d entries). This is needed to provide configs
       before new packages are installed, so dphys-admin can pretend that  the
       packages	 were already once installed (and then non-purge removed), and
       so prevent questions on install,	which is required for  unattended  in-
       stalls.	[Note  that this pretending does not go	as far as setting deb-
       conf up.	Broken packages	that ignore config files and only look at deb-
       conf will still ask questions.]

       As  result of this, when	installing for the first time on an new	system
       (such as	installing Debian by the dphys3	end2stage feature,  which  in-
       stalls  first dphys-config and then dphys-admin), any scripts installed
       by packages by dphys-admin, to be called	on config file	install/update
       will still be missing, and so not runnable. Either ignore the warnings,
       or better call the scripts by something like this:

       file:place:if [ -x script ]; then script; fi

       Note that in this case, trying to run dphys-config for  a  second  time
       after dphys-admin has installed packages	and scripts, will not automat-
       ically mend this,  as  the  config  files  have	not  changed,  and  so
       dphys-config  will  not (re-)run	their scripts. Therefore packages con-
       taining such scripts must also, as part of their	 postinst  (or	init.d
       which  is called	by postinst), check for	existing config	files and then
       run their scripts. This is the normal behaviour of quite	a few packages
       anyway.	Of course this requires	the scripts to be idempotent, which is
       official	Debian policy anyway.


D-PHYS Configuration Tools	  2010.01.21		       DPHYS-CONFIG(1)


Want to link to this manual page? Use this URL:

home | help