Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
RDIFFDIR(1)			 User Manuals			   RDIFFDIR(1)

NAME
       duplicity - Encrypted backup using rsync	algorithm

SYNOPSIS
       duplicity [options] source_directory target_url

       duplicity [options] source_url target_directory

       duplicity full [options]	source_directory target_url

       duplicity incremental [options] source_directory	target_url

       duplicity restore [options] source_url target_directory

       duplicity verify	[options] source_url target_directory

       duplicity collection-status [options] target_url

       duplicity list-current-files [options] target_url

       duplicity cleanup [options] target_url

       duplicity remove-older-than time	[options] target_url

       duplicity remove-all-but-n-full count [options] target_url

DESCRIPTION
       Duplicity incrementally backs up	files and directory by encrypting tar-
       format volumes with GnuPG and uploading them to	a  remote  (or	local)
       file  server.   Currently  local, ftp, ssh/scp, rsync, WebDAV, WebDAVs,
       HSi and Amazon S3  backends  are	 available.   Because  duplicity  uses
       librsync,  the incremental archives are space efficient and only	record
       the parts of files that have changed since the last backup.   Currently
       duplicity  supports  deleted files, full	unix permissions, directories,
       symbolic	links, fifos, etc., but	not hard links.

       Duplicity will read the PASSPHRASE environment  variable	 to  find  the
       passphrase  to  give  to	 GnuPG.	  If this is not set, the user will be
       prompted	for the	passphrase.

       If you are backing up the  root	directory  /,  remember	 to  --exclude
       /proc,  or  else	 duplicity  will  probably crash on the	weird stuff in
       there.

EXAMPLES
       Here is an example of a backup,	using  scp  to	back  up  /home/me  to
       some_dir	on the other.host machine:

	      duplicity	/home/me scp://uid@other.host/some_dir

       If  the	above  is run repeatedly, the first will be a full backup, and
       subsequent ones will be incremental.  To	force a	full backup,  use  the
       full action:

	      duplicity	full /home/me scp://uid@other.host/some_dir

       Now  suppose we accidentally delete /home/me and	want to	restore	it the
       way it was at the time of last backup:

	      duplicity	scp://uid@other.host/some_dir /home/me

       Duplicity enters	restore	mode because the URL comes  before  the	 local
       directory.   If	we  wanted  to restore just the	file "Mail/article" in
       /home/me	as it was three	days ago into /home/me/restored_file:

	      duplicity	    -t	   3D	   --file-to-restore	  Mail/article
	      scp://uid@other.host/some_dir /home/me/restored_file

       The  following command compares the files we backed up, so see what has
       changed since then:

	      duplicity	verify scp://uid@other.host/some_dir /home/me

       Finally,	duplicity recognizes  several  include/exclude	options.   For
       instance,  the  following  will	backup the root	directory, but exclude
       /mnt, /tmp, and /proc:

	      duplicity	 --exclude  /mnt  --exclude  /tmp  --exclude  /proc  /
	      file:///usr/local/backup

       Note  that  in  this  case  the	destination  is	 the  local  directory
       /usr/local/backup.  The following will backup only the /home  and  /etc
       directories under root:

	      duplicity	 --include  /home  --include  /etc  --exclude  '**'  /
	      file:///usr/local/backup

       Duplicity can also access a repository via ftp.	 If  a	user  name  is
       given,  the  environment	variable FTP_PASSWORD is read to determine the
       password:

	      FTP_PASSWORD=mypassword		duplicity	    /local/dir
	      ftp://user@other.host/some_dir

ACTIONS
       cleanup
	      Delete  the  extraneous  duplicity  files	 on the	given backend.
	      Non-duplicity files, or files in complete	data sets will not  be
	      deleted.	 This  should only be necessary	after a	duplicity ses-
	      sion fails or is aborted prematurely.

       collection-status
	      Summarize	the status of the backup repository  by	 printing  the
	      chains and sets found, and the number of volumes in each.

       full   Indicate	full backup.  If this is set, perform full backup even
	      if signatures are	available.

       incr   If this is requested an incremental backup  will	be  performed.
	      Duplicity	 will  abort  if  old signatures cannot	be found.  The
	      default is to switch to full backup under	these conditions.

       list-current-files
	      Lists the	files currently	backed up in the archive.  The	infor-
	      mation  will  be extracted from the signature files, not the ar-
	      chive data itself.  Thus the whole archive does not have	to  be
	      downloaded,  but	on  the	 other	hand  if  the archive has been
	      deleted or corrupted, this command may not detect	it.

       remove-older-than time
	      Delete all backup	sets older than	the given time.	 If old	backup
	      sets  will  not be deleted if backup sets	newer than time	depend
	      on them.	See the	TIME FORMATS  section  for  more  information.
	      Note,  this  action  cannot  be  combined	 with  backup or other
	      actions, such as cleanup.

       remove-all-but-n-full count
	      Delete all backups sets that are older than  the	count:th  last
	      full  backup  (in	 other words, keep the last count full backups
	      and associated incremental sets).	 count	must  be  larger  than
	      zero. A value of 1 means that only the single most recent	backup
	      chain will be kept.

       verify Enter verify mode	instead	of restore.  If	the  --file-to-restore
	      option  is  given,  restrict  verify  to that file or directory.
	      duplicity	will exit with a non-zero error	level if any files are
	      different.   On  verbosity  level	4 or higher, log a message for
	      each file	that has changed.

OPTIONS
       --allow-source-mismatch
	      Do not abort on attempts to use the same archive dir  or	remote
	      backend  to  back	up different directories.  duplicity will tell
	      you if you need this switch.

       --archive-dir path
	      When backing up or restoring, specify the	local  archive	direc-
	      tory.   This  option is not necessary, but if hash data is found
	      locally in path it will be used in preference to the remote hash
	      data. Use	of this	option does not	imply that the archive data is
	      no longer	stored in the backup destination, nor that  the	 local
	      archive directory	need be	kept safe. The local archive directory
	      is a performance optimization only, and may safely be  discarded
	      at any time.

       --encrypt-key key
	      When  backing  up,  encrypt  to the given	public key, instead of
	      using symmetric (traditional) encryption.	 Can be	specified mul-
	      tiple times.

       --exclude shell_pattern
	      Exclude the file or files	matched	by shell_pattern.  If a	direc-
	      tory is matched, then files under	that directory	will  also  be
	      matched.	See the	FILE SELECTION section for more	information.

       --exclude-device-files
	      Exclude  all device files.  This can be useful for security/per-
	      missions reasons or if rdiff-backup is not handling device files
	      correctly.

       --exclude-filelist filename
	      Excludes	the  files listed in filename.	See the	FILE SELECTION
	      section for more information.

       --exclude-filelist-stdin
	      Like --exclude-filelist, but the list of files will be read from
	      standard	input.	See the	FILE SELECTION section for more	infor-
	      mation.

       --exclude-globbing-filelist filename
	      Like --exclude-filelist but each line of the  filelist  will  be
	      interpreted  according  to  the  same  rules  as	--include  and
	      --exclude.

       --exclude-other-filesystems
	      Exclude files on file  systems  (identified  by  device  number)
	      other  than  the file system the root of the source directory is
	      on.

       --exclude-regexp	regexp
	      Exclude files matching the given regexp.	Unlike	the  --exclude
	      option,  this  option  does  not	match  files in	a directory it
	      matches.	See the	FILE SELECTION section for more	information.

       --file-to-restore path
	      This option may be given in restore mode,	causing	only  path  to
	      be  restored  instead  of	 the entire contents of	the backup ar-
	      chive.  path should be given relative to the root	of the	direc-
	      tory backed up.

       --full-if-older-than time
	      Perform a	full backup if an incremental backup is	requested, but
	      the latest full backup in	the collection is older	than the given
	      time.  See the TIME FORMATS section for more information.

       --force
	      Proceed  even if data loss might result.	Duplicity will let the
	      user know	when this option is required.

       --ftp-passive
	      Use passive (PASV) data connections.  The	default	is to use pas-
	      sive, but	to fallback to regular if the passive connection fails
	      or times out.

       --ftp-regular
	      Use regular (PORT) data connections.

       --gpg-options options
	      Allows you to pass options to gpg	encryption.  The options  list
	      should  be  of the form "opt1=parm1 opt2=parm2" where the	string
	      is quoted	and the	only spaces allowed are	between	options.

       --include shell_pattern
	      Similar to --exclude but include matched files instead.	Unlike
	      --exclude,  this	option	will  also match parent	directories of
	      matched files (although not necessarily  their  contents).   See
	      the FILE SELECTION section for more information.

       --include-filelist filename
	      Like  --exclude-filelist,	 but include the listed	files instead.
	      See the FILE SELECTION section for more information.

       --include-filelist-stdin
	      Like --include-filelist, but read	the  list  of  included	 files
	      from standard input.

       --include-globbing-filelist filename
	      Like  --include-filelist	but  each line of the filelist will be
	      interpreted  according  to  the  same  rules  as	--include  and
	      --exclude.

       --include-regexp	regexp
	      Include  files  matching	the  regular  expression regexp.  Only
	      files explicitly matched by regexp  will	be  included  by  this
	      option.  See the FILE SELECTION section for more information.

       --no-encryption
	      Do  not  use  GnuPG  to encrypt files on remote system.  Instead
	      just write gzipped volumes.

       --no-print-statistics
	      By default duplicity will	print  statistics  about  the  current
	      session  after  a	 successful backup.  This switch disables that
	      behavior.

       --null-separator
	      Use nulls	(\0) instead of	 newlines  (\n)	 as  line  separators,
	      which  may help when dealing with	filenames containing newlines.
	      This affects the expected	format of the files specified  by  the
	      --{include|exclude}-filelist[-stdin]  switches  as  well	as the
	      format of	the directory statistics file.

       --num-retries number
	      Number of	retries	to make	on errors before giving	up.

       --scp-command command
	      This option only matters when using the  ssh/scp	backend.   The
	      command  will  be	 used instead of scp to	send or	receive	files.
	      The default is "scp". To list and	 delete	 existing  files,  the
	      sftp command is used.  See --ssh-options and --sftp-command.

       --sftp-command command
	      This  option  only  matters when using the ssh/scp backend.  The
	      command will be used instead of sftp for	listing	 and  deleting
	      files.  The default is "sftp". File transfers are	done using the
	      scp command. See --ssh-options and --scp-command.

       --sign-key key
	      This option can be used when  backing  up	 or  restoring.	  When
	      backing  up,  all	 backup	 files	will be	signed with keyid key.
	      When restoring, duplicity	will signal an	error  if  any	remote
	      file  is	not  signed  with the given keyid.  key	should be an 8
	      character	hex string, like AA0E73D2.

       --ssh-askpass
	      Tells the	ssh/scp	backend	to use FTP_PASSWORD from the  environ-
	      ment,  or,  if  that  is not present, to prompt the user for the
	      remote system password.

       --ssh-options options
	      Allows you to pass options to  the  ssh/scp/sftp	backend.   The
	      options list should be of	the form "opt1=parm1 opt2=parm2" where
	      the option string	is quoted and  the  only  spaces  allowed  are
	      between  options.	 The  option string will be passed verbatim to
	      both scp and sftp, whose command line syntax  differs  slightly:
	      options  passed  with --ssh-options should therefore be given in
	      the long option format described in ssh_config(5), like in  this
	      example:

	      duplicity		--ssh-options="-oProtocol=2	   -oIdentity-
	      File=/my/backup/id" /home/me scp://uid@other.host/some_dir

       --short-filenames
	      If this option is	specified, the names of	 the  files  duplicity
	      writes will be shorter (about 30 chars) but less understandable.
	      This may be useful when backing up to MacOS or another OS	or  FS
	      that doesn't support long	filenames.

       --tempdir directory
	      Use  this	 existing  directory  for  duplicity  temporary	 files
	      instead of the system default, which is usually the /tmp	direc-
	      tory. This option	supercedes any environment variable.

       -ttime, --restore-time time
	      When restoring, specify the time to restore to.

       --time-separator	char
	      Use  char	 as  the  time separator in filenames instead of colon
	      (":").

       -v[0-9],	--verbosity [0-9]
	      Specify verbosity	level (0 is total silent, 3  is	 the  default,
	      and 9 is noisiest).

       --version
	      Print duplicity's	version	and quit.

       --volsize number
	      Change the volume	size to	number Mb. Default is 5Mb.

URL FORMAT
       Duplicity  tries	to maintain a standard URL format as much as possible.
       The generic format for a	URL is:

	      scheme://user[:password]@host[:port]/[/]path

       It is not recommended to	expose the password on the command line	 since
       it could	be revealed to anyone with permissions to do process listings,
       however,	it is permitted.

       In protocols that support it, the path may be  preceeded	 by  a	single
       slash,  '/path',	to represent a relative	path to	the target home	direc-
       tory, or	preceeded by a double slash, '//path', to represent  an	 abso-
       lute filesystem path.

       Formats of each of the URL schemes follow:

	      ssh://user[:password]@other.host[:port]/some_dir

	      scp://user[:password]@other.host[:port]/some_dir

	      ftp://user[:password]@other.host[:port]/some_dir

	      hsi://user[:password]@other.host/some_dir

	      file:///some_dir

	      rsync://user[:password]@other.host[:port]::/module/some_dir

	      rsync://user[:password]@other.host[:port]/relative_path

	      rsync://user[:password]@other.host[:port]//absolute_path

	      s3://host/bucket_name[/prefix]

	      s3+http://bucket_name[/prefix]

	      webdav://user[:password]@other.host/some_dir

	      webdavs://user[:password]@other.host/some_dir

TIME FORMATS
       duplicity  uses time strings in two places.  Firstly, many of the files
       duplicity creates will have the time in their filenames in the w3 date-
       time  format  as	 described  in a w3 note at http://www.w3.org/TR/NOTE-
       datetime.  Basically they look like "2001-07-15T04:09:38-07:00",	 which
       means  what it looks like.  The "-07:00"	section	means the time zone is
       7 hours behind UTC.

       Secondly, the -t, and --restore-time options take a time	string,	 which
       can be given in any of several formats:

       1.     the string "now" (refers to the current time)

       2.     a	 sequences of digits, like "123456890" (indicating the time in
	      seconds after the	epoch)

       3.     A	string like "2002-01-25T07:00:00+02:00"	in datetime format

       4.     An interval, which is a number followed by one of	the characters
	      s,  m,  h,  D,  W,  M, or	Y (indicating seconds, minutes,	hours,
	      days, weeks, months, or years respectively), or a	series of such
	      pairs.  In this case the string refers to	the time that preceded
	      the current time by the length of	the interval.	For  instance,
	      "1h78m" indicates	the time that was one hour and 78 minutes ago.
	      The calendar here	is unsophisticated: a month is always 30 days,
	      a	year is	always 365 days, and a day is always 86400 seconds.

       5.     A	date format of the form	YYYY/MM/DD, YYYY-MM-DD,	MM/DD/YYYY, or
	      MM-DD-YYYY, which	indicates midnight on  the  day	 in  question,
	      relative	to  the	 current  timezone  settings.	For  instance,
	      "2002/3/5", "03-05-2002",	and "2002-3-05"	all  mean  March  5th,
	      2002.

FILE SELECTION
       duplicity  accepts  the	same file selection options rdiff-backup does,
       including --exclude, --exclude-filelist-stdin, etc.

       When duplicity is run, it searches through the given  source  directory
       and backs up all	the files specified by the file	selection system.  The
       file selection system comprises a number	of file	selection  conditions,
       which  are  set	using  one  of	the  following	command	 line options:
       --exclude,   --exclude-device-files,   --exclude-filelist,   --exclude-
       filelist-stdin,	    --exclude-globbing-filelist,     --exclude-regexp,
       --include,  --include-filelist,	--include-filelist-stdin,   --include-
       globbing-filelist, and --include-regexp.	 Each file selection condition
       either matches or doesn't match a given file.  A	given file is excluded
       by  the	file  selection	 system	 exactly  when the first matching file
       selection condition specifies that the file be excluded;	otherwise  the
       file is included.

       For instance,

	      duplicity	    --include	  /usr	   --exclude	 /usr	  /usr
	      scp://user@host/backup

       is exactly the same as

	      duplicity	/usr scp://user@host/backup

       because the include and	exclude	 directives  match  exactly  the  same
       files, and the --include	comes first, giving it precedence.  Similarly,

	      duplicity	--include  /usr/local/bin  --exclude  /usr/local  /usr
	      scp://user@host/backup

       would  backup  the /usr/local/bin directory (and	its contents), but not
       /usr/local/doc.

       The include, exclude, include-globbing-filelist,	and  exclude-globbing-
       filelist	 options  accept extended shell	globbing patterns.  These pat-
       terns can contain the special patterns *, **, ?,	and [...].   As	 in  a
       normal  shell,  *  can be expanded to any string	of characters not con-
       taining "/", ?  expands to any character	except "/", and	[...]  expands
       to a single character of	those characters specified (ranges are accept-
       able).  The new special pattern,	**, expands to any string  of  charac-
       ters  whether  or  not  it  contains  "/".  Furthermore,	if the pattern
       starts with "ignorecase:" (case insensitive), then this prefix will  be
       removed	and any	character in the string	can be replaced	with an	upper-
       or lowercase version of itself.

       Remember	that you may need to quote these characters when  typing  them
       into  a	shell,	so  the	shell does not interpret the globbing patterns
       before duplicity	sees them.

       The --exclude pattern option matches a file iff:

       1.     pattern can be expanded into the file's filename,	or

       2.     the file is inside a directory matched by	the option.

       Conversely, --include pattern matches a file iff:

       1.     pattern can be expanded into the file's filename,

       2.     the file is inside a directory matched by	the option, or

       3.     the file is a directory which contains a	file  matched  by  the
	      option.

       For example,

	      --exclude	/usr/local

       matches /usr/local, /usr/local/lib, and /usr/local/lib/netscape.	 It is
       the same	as --exclude /usr/local	--exclude '/usr/local/**'.

	      --include	/usr/local

       specifies    that     /usr,     /usr/local,     /usr/local/lib,	   and
       /usr/local/lib/netscape	(but not /usr/doc) all be backed up.  Thus you
       don't have to worry about including parent  directories	to  make  sure
       that included subdirectories have somewhere to go.  Finally,

	      --include	ignorecase:'/usr/[a-z0-9]foo/*/**.py'

       would  match  a	file  like  /usR/5fOO/hello/there/world.py.  If	it did
       match anything, it would	also match /usr.  If there is no existing file
       that  the given pattern can be expanded into, the option	will not match
       /usr.

       The --include-filelist,	--exclude-filelist,  --include-filelist-stdin,
       and --exclude-filelist-stdin options also introduce file	selection con-
       ditions.	 They direct duplicity to read in a file, each line  of	 which
       is  a file specification, and to	include	or exclude the matching	files.
       Lines are separated by newlines or  nulls,  depending  on  whether  the
       --null-separator	 switch	 was given.  Each line in a filelist is	inter-
       preted similarly	to the way extended shell patterns  are,  with	a  few
       exceptions:

       1.     Globbing patterns	like *,	**, ?, and [...]  are not expanded.

       2.     Include  patterns	 do  not  match	 files	in a directory that is
	      included.	 So /usr/local in  an  include	file  will  not	 match
	      /usr/local/doc.

       3.     Lines  starting with "+ "	are interpreted	as include directives,
	      even if found in a filelist  referenced  by  --exclude-filelist.
	      Similarly,  lines	 starting with "- " exclude files even if they
	      are found	within an include filelist.

       For example, if file "list.txt" contains	the lines:

	      /usr/local
	      -	/usr/local/doc
	      /usr/local/bin
	      +	/var
	      -	/var

       then "--include-filelist	list.txt" would	include	/usr, /usr/local,  and
       /usr/local/bin.	      It       would	  exclude      /usr/local/doc,
       /usr/local/doc/python,  etc.   It   neither   excludes	nor   includes
       /usr/local/man, leaving the fate	of this	directory to the next specifi-
       cation condition.  Finally, it is undefined what	happens	with /var.   A
       single file list	should not contain conflicting file specifications.

       The --include-globbing-filelist and --exclude-globbing-filelist options
       also specify filelists, but each	line in	the filelist  will  be	inter-
       preted  as  a  globbing pattern the way --include and --exclude options
       are interpreted (although "+ " and "- " prefixing  is  still  allowed).
       For instance, if	the file "globbing-list.txt" contains the lines:

	      dir/foo
	      +	dir/bar
	      -	**

       Then  "--include-globbing-filelist  globbing-list.txt" would be exactly
       the same	as specifying "--include dir/foo --include  dir/bar  --exclude
       **" on the command line.

       Finally,	 the  --include-regexp	and --exclude-regexp allow files to be
       included	and excluded if	their filenames	match a	python regular expres-
       sion.   Regular	expression  syntax is too complicated to explain here,
       but is covered in Python's library reference.  Unlike the --include and
       --exclude  options,  the	 regular  expression options don't match files
       containing or contained in matched files.  So for instance

	      --include	'[0-9]{7}(?!foo)'

       matches any files whose full pathnames  contain	7  consecutive	digits
       which  aren't followed by 'foo'.	 However, it wouldn't match /home even
       if /home/ben/1234567 existed.

OPERATION AND DATA FORMATS
       This section describes duplicity's basic	operation and  the  format  of
       its  data  files.   It should not necessary to read this	section	to use
       duplicity.

       The files used by duplicity to store backup data	are  tarfiles  in  GNU
       tar  format.   They  can	be produced independently by rdiffdir(1).  For
       incremental backups, new	files are saved	normally in the	tarfile.   But
       when  a	file  changes, instead of storing a complete copy of the file,
       only a diff is stored, as generated by rdiff(1).	 If a file is deleted,
       a  0  length  file  is  stored in the tar.  It is possible to restore a
       duplicity archive "manually" by using tar and then cp, rdiff, and rm as
       necessary.  These duplicity archives have the extension difftar.

       Both full and incremental backup	sets have the same format.  In effect,
       a full backup set is an incremental one generated from an empty	signa-
       ture  (see  below).   The  files	 in  full  backup sets will start with
       duplicity-full while the	incremental  sets  start  with	duplicity-inc.
       When  restoring,	 duplicity  applies patches in order, so deleting, for
       instance, a full	backup set may make related  incremental  backup  sets
       unuseable.

       In  order  to determine which files have	been deleted, and to calculate
       diffs for changed files,	duplicity needs	to process  information	 about
       previous	 sessions.  It stores this information in the form of tarfiles
       where each entry's data contains	the signature (as produced  by	rdiff)
       of  the file instead of the file's contents.  These signature sets have
       the extension sigtar.

       Signature files are not required	to restore a backup set,  but  without
       an  up-to-date signature, duplicity cannot append an incremental	backup
       to an existing archive.

       To save bandwidth, duplicity generates full signature sets  and	incre-
       mental signature	sets.  A full signature	set is generated for each full
       backup, and an incremental one  for  each  incremental  backup.	 These
       start   with   duplicity-full-signatures	 and  duplicity-new-signatures
       respectively.  If --archive-dir	is  used,  these  signatures  will  be
       stored  both  locally  and  remotely.   The  remote  signatures will be
       encrypted if encryption is enabled.  The	local signatures will  not  be
       encrypted

ENVIRONMENT VARIABLES
       TMPDIR, TEMP, TMP
	      In  decreasing  order  of	importance, specifies the directory to
	      use for temporary	files (inherited from Python's	tempfile  mod-
	      ule).

BUGS
       Hard  links  currently  unsupported (they will be treated as non-linked
       regular files).

       Bad signatures will be treated as empty instead of logging  appropriate
       error message.

AUTHOR
       Original	Author - Ben Escoto <bescoto@stanford.edu>

       Current Maintainer - Kenneth Loafman <kenneth@loafman.com>

SEE ALSO
       rdiffdir(1), python(1), rdiff(1), rdiff-backup(1).

Version	0.4.12			   July	2008			   RDIFFDIR(1)

NAME | SYNOPSIS | DESCRIPTION | EXAMPLES | ACTIONS | OPTIONS | URL FORMAT | TIME FORMATS | FILE SELECTION | OPERATION AND DATA FORMATS | ENVIRONMENT VARIABLES | BUGS | AUTHOR | SEE ALSO

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=duplicity&manpath=FreeBSD+7.1-RELEASE+and+Ports>

home | help