Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages


home | help
RDUP(1)				     rdup			       RDUP(1)

       rdup - generate a file list suitable for	making backups

       rdup [-N	timestamp] -[-P	CMD]...	 [OPTION]...  FILELIST [DIR/FILE]...

       rdup is a utility inspired by rsync and the Plan9 way of	doing backups.
       rdup itself does	not backup anything. It	only prints a  list  of	 files
       that are	changed, or all	files in case of a null	dump.  It also handles
       files that are removed, allowing	for correct incremental	 backups.  All
       paths printed are absolute. rdup	uses the change	time (ctime) to	decide
       whether a file is altered.

       It works	as follows, for	a full dump

       1.     Crawl all	directories, and print all the names found to standard
	      output.	Also  check for	._rdup_. files to restore the original
	      ownership	(see the -u flag).

       2.     Write a filelist with all	the names found	 when  crawling.   Use
	      this list	to calculate the correct incremental dump.

       And for incremental dumps

       1.     Read in the filelist that	was written when doing a full dump.

       2.     Touch the	time stamp file.

       3.     Crawl all	the directories	again.

       4.     Diff 1. and 2. to	get two	lists; one of removed items and	one of
	      added/modified items.

       5.     Write the	removed	items to standard output

       6.     Write the	modified/new items to standard output.

       7.     Write a new filelist.

       The FILELIST is a internal list rdup writes to, to keep track of	 which
       files  are  in  a  backup.  If  you  don't  want	this (i.e. make	a full
       backup),	use /dev/null here. The	file /dev/null is handled specially by
       rdup: if	detected no new	file list is written.

       The  DIRS/FILES can be specified	multiple times.	These are the directo-
       ries and	files you want to backup. If omitted it	defaults to  the  cur-
       rent directory "." .

       If  the	-N timestamp option is not given, all paths found are printed.
       Only when a -N timestamp	file is	given, times can be  compared  and  an
       incremental output can be generated.

       rdup  prints  a	filelist to standard output.  Subsequent programs in a
       pipe line can be	used to	actually implement to backup scheme.  After  a
       run  a new FILELIST is written. No warning is given when	FILELIST is an
       existing	file, it just gets overwritten by rdup.	New  runs  will	 print
       out  only  those	 files that have actually changed or are removed since
       the last	run, thereby making incremental	backups	possible.

       Files are checked for changes by	comparing the c-time (change time), if
       this  time  is  NEWER than the c-time of	timestamp file the pathname is
       printed to standard output.  When  files	 are  removed  they  are  also
       printed	to standard output, but	they are prefixed with a '-'. See FOR-
       MAT below. The default format rdup uses is: "%p%T %b %t %u %U %g	%G  %l

       Note,  that rdup	also supports hashing of files,	this makes it possible
       to check	the local hash with the	hash of	the backed up file.

       All errors are written to standard error.  If  the  directory  or  file
       does not	exist, they are	skipped	and a warning is emitted.

       The  general  idea is to	be very	UNIX like and create a bunch of	simple
       programs	which each do a	their specific thing very well.	With rdup  and
       a  small	 shell	script (50 lines) one can implement encrypted and com-
       pressed backups.

       As rdup doesn't backup anything,	the backup policy;  what  you  backup,
       how  you	 backup,  how  often  and  how you restore; is all left	to the
       scripts and your	imagination. To	kick start your	imagination see	 rdup-
       tr(1), rdup-up(1) and maybe rdup-backups.

       -P command
	      Filter all output	through	command, multiple -P's can be used.

	      Due  to  the nature of pipes in Unix, this pipeline is recreated
	      for every	file processed.	Also see 'Child	Processes' below.

       -F format
	      Specify a	printf-style format to use. See	FORMAT below.

       -N timestamp
	      use the c_time of	file timestamp as the timestamp	to decide what
	      to include in the	incremental backup list. If timestamp does not
	      exist a full dump	is performed.

       -M timestamp
	      As -N, but look at the m_time of timestamp.

       -R     Reverse the output of rdup. Tools	accepting this ouput must cre-
	      ate  leading  directory  as  they	see them. This option allows a
	      script --	running	as a normal user -- to put files in  a	direc-
	      tory which could have 0600 as its	permission.

       -E file
	      The file named 'file' contains a list of Perl-compatible regular
	      expressions (PCRE), one per line,	that rdup will use to  exclude
	      names.  A	 '#'  at the start of the line can be used to signal a
	      comment.	Empty lines are	ignored.

	      If a directory is	excluded, rdup won't descend  in  that	direc-
	      tory, so all files in that directory are also excluded.

	      The  directories leading up to the directory to be backed	up can
	      not be excluded. If you use a command line like:

		      rdup /dev/null /home/miekg/bin

	      The directories '/home',	'/home/miekg',	'/home/miekg/bin'  are
	      always printed.

	      If  you want to exclude the file '/home/miekg/blaat' you need to
	      add the following	regular	expression: '/home/miekg/blaat'.

	      If you want to exclude all  .mozilla/cache  directories  of  all
	      users you	can use	'/home/.*/.mozilla/cache/.*'. This doesn't ex-
	      clude the	directory itself and I'm assuming that the users' home
	      directories are found under '/home'.

	      Also  note  that rdup does not print directories with a trailing

       -a     Restore the original access times	on files and directories.

       -n     Don't honor .nobackup files. Normally if such a  file  is	 found
	      the  directory  and  all files containing	it, are	not printed to
	      standard output. Now they	are.

       -u     Output files with	a ._rdup_. prefix just as normal  files.  When
	      this option is not given these files are skipped by rdup.

       -r     Only  print  removed  files; entries that	start with a `-'. This
	      option unsets -m.

       -m     Only print modified/new files; entries that start	 with  a  `+'.
	      This option unsets -r.

       -v     Be  more	verbose.  When	used each path will also be printed to
	      standard error.

       -s size
	      Don't output files larger	than size bytes.  This can be used  to
	      limit  the  amount of data to be transferred when	doing a	remote
	      backup.  This option only	applies	to files.

       -x     Stay on the local	filesystem.

       -V     Print rdup's version.

       -h     Give an overview of the options.

   Child Processes (-P flag)
       When creating output you	might also want	to 'pipe' the contents of each
       file  through  a	 number	 of commands, say a compression	and encryption
       utility.	Note that this is different than compressing  the  entire  ar-
       chive  as GNU tar allows	by using the -z	option.	 So this is where rdup
       comes in. It allows you to create a normal archive in which  each  file
       is  encrypted (or compressed.  reversed or whatever). rdup does this by
       forking child processes which transform the content.

       If one of the forked children returns an	exit code other	than zero (0),
       it  is  assumed	the whole conversion process failed. In	that case rdup

       As said rdup works by forking off a number of  child  processes	(those
       commands	 named	with  the  -P  option(s)),  interconnecting these with
       pipes. The current file is connected to the first  child.   The	output
       created by these	child processes	is captured by the parent (rdup).  The
       contents	is then	written	to standard output in an archive format.  As a
       picture	says  more than	a thousand words here is an ASCII image	of the

			  +--- ...   (stdout)	 ... ----> archive
		     rdup  <---	...   ... <----+
		 loop #files		       |
		file ---> cmd1 | cmd2 |	...| cmdN

	       rm -f timestamp && rdup -N timestamp LIST DIR

       A full-dump filelist is printed to standard output. And with:

	       rdup -N timestamp LIST DIR

       An incremental dump filelist is printed.	The file timestamp is used  to
       save  the  exact	time of	rdup's run. The	file LIST is used to calculate
       the correct incremental dump list, this is needed for  files  that  are
       removed,	or have	a different type.

       If backups are made by a	non-root user or on a filesystem that does not
       implement/allow chown (think sshfs mounted by an	ordinary  user),  rdup
       creates	a seperate file	which stores the correct user and group	infor-
       mation. If the file's name is foobar a new file	called	._rdup_.foobar
       is  created in the same directory. This file contains one line, for in-


       That tells that the actual ownership should be root:root. For  directo-
       ries a ._rdup_. file is created inside the current directory.

       The default format rdup uses is:	"%p%T %b %t %u %U %g %G	%l %s\n%n%C"

       The following escape sequences are understood by	rdup:

	       'p': '+'	if file	is new/modified, '-' if	removed
	       'b': permission bits from lstat(2), octal in four digits
	       'm': the	file mode bits,	st_mode	from lstat(2), decimal digits
	       'u': uid
	       'U': username
	       'g': gid
	       'G': groupname
	       'l': path name length
	       's': original file size,	but see	CAVEATS
	       'n': path name
	       'N':  path  name,  but  in case of a soft- or hardlink only the
       link name
	       't': time of modification (seconds from epoch)
	       'H': the	SHA1 hash of the file, all zeros ("0") for  all	 other
	       'T': file type
		     - normal file, l symlink, h hardlink, d directory,
		     c character device, b block device, p named pipe
		     and s socket
	       'C': the	content	of the file (none for all other	types)

       To delimit the output of	rdup with NULLs	you can	use '\0' in the	format

       Any file	content	is written in a	block/chunk  based  manner.  The  last
       block  is  signaled with	a null block. A	block start entry is ASCII and
       is formatted as follows:	VVBLOCKBBBBB\n .  Where	'VV' is	 the  version,
       currently  at '01', then	the literal string 'BLOCK' and then the	amount
       of bytes	(BBBBB), typical '08192'. And then a newline.  This look  like

	       <START OF THE FIRST 8192	BYTES>01BLOCK00015
	       <ANOTHER	15 BYTES>01BLOCK00000

       A byte count of zero signals a stop block.

       rdup writes the (internal) FILELIST in the following format:


       Where  MODE  is the st_mode from	stat(2), DEV is	the dev	id as returned
       by the stat call	and INODE is the inode number -	rdup needs  this  info
       to  decide  if  a  directory  is	 renamed.  LINK	 is  equal  to 'h' for
       hardlinks, 'l' for symlinks and otherwise it is '*'.  UID and  GID  are
       the  numeric  user and group id of the file. PATH_SIZE is the length of
       PATH. FILE_SIZE the file	size.  And finally PATH	is  the	 path  of  the

       A typical example is:

	      16893	2050	 32085	   *	 1000	  1000	   30	  4096

       The output generated by rdup is formatted like:


       This makes it possible possible for a remote shell  script  to  receive
       the actual file contetns	and make a backup.

       For directories:	the FILE_SIZE is zero and no content is	printed. Thus:

	       +d 0755 1260243445 1000 miekg 1000 miekg	11 0\n

       For regular files the following is a sample output:

	       +- 0644 1260243445 1000 miekg 1000 miekg	32 6\n

       Where aaa/a is a	regular	file containing	the word 'hello\n'

       Soft- and hardlinks are handled differently when	using %n, if you don't
       like this behavior use %N.  The PATH name is generated from the	link's
       name and	its target. A symlink like

	       /home/bin/blaat -> /home/bin/bliep

       is  printed  as	'/home/bin/blaat -> /home/bin/bliep'. The PATH_SIZE is
       modified	accordingly, where ' ->	' (4 characters) is also counted.  The
       FILE_SIZE is not	needed for soft- or hardlinks, so it is	set the	length
       of the link's name -- the part left of the '  ->',  in  this  case  the
       length of '/home/bin/blaat'.

       If  rdup	 encounters  a hardlink	it is handled in the same way, but the
       output type is set to 'h' instead of 'l'. A hardlink is	only  detected
       if  rdup	finds a	file with the same inode and device number as a	previ-
       ous one,	i.e. such hardlinks must be contained in your backup.

       Again note: with	'%N' only the link's name is printed. The FILE_SIZE is
       still set to the	length of the link's name.

   Device Files
       For devices the size field (%s) is changed to hold the major,minor num-
       ber of the device. So if	a major	number is 8 and	the minor number is  0
       (under  Linux  this is /dev/sda), its size will be 8,0. The numbers are
       only separated with a comma `,'.

       You will	probably think rdup will descend into the directory  the  sym-
       bolic  link  points  to.	 This  is not what actually happens, rdup will
       print any directories leading up	to the symlink and  will  not  descend
       into the	directory. GNU tar works the same.

       rdup return a zero exit code on success,	otherwise 1 is returned.  rdup
       will abort if a file can	not be concatenated, if	a  regular  expression
       can not be compiled or if a signal is received.

       The  next  set  of examples will	all make a full	dump --	because	of the
       use of /dev/null. See rdup-tr(1)	for more advanced examples.

   rdup	(mirroring)
	       rdup /dev/null ~/bin | rdup-up -t /shared/backup
	      rdup /dev/null /shared/backup | rdup-up -t /tmp/restore
	      cp -rap /shared/backup /tmp/restore

   rdup	(archiving)
	      rdup /dev/null ~/bin > my-archive.rdup
	      rdup-up -t /tmp/restore <	my-archive.rdup

	      rdup -R -F '%N\n'	/dev/null ~/bin	 |  cpio  -o  -Hcrc  >	my-ar-
	      cpio -i -d -Hcrc < my-archive.cpio

	      rdup  -F	'%N\n'	/dev/null ~/bin	| tar c	-f my-archive.tar -T -
	      tar x -f my-archive.tar

       Written by Miek Gieben.

       Report bugs to <>.

       http:/ is the main  site  of  rdup.  Also  see
       rdup-tr(1), rdup-up(1) and rdup-backups(7).

       Copyright (C) 2005-2011 Miek Gieben. This is free software. There is NO
       warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PUR-

       Licensed	 under	the  GPL version 3. See	the file LICENSE in the	source
       distribution of rdup.

1.1.14				  24 Dec 2005			       RDUP(1)


Want to link to this manual page? Use this URL:

home | help