Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
DOVEADM-SYNC(1)			    Dovecot		       DOVEADM-SYNC(1)

NAME
       doveadm-sync - Dovecot's	two-way	mailbox	synchronization	utility
       doveadm-backup -	Dovecot's one-way mailbox synchronization utility

SYNOPSIS
       doveadm	[-Dv]  sync [-u	user|-A|-F file] [-S socket_path] [-1fPRU] [-l
       secs] [-r rawlog_path] [-m mailbox] [-g mailbox_guid] [-n namespace|-N]
       [-x exclude] [-s	state] [-T secs] [-t start date] [-e end date] -d|des-
       tination

       doveadm [-Dv] backup [-u	user|-A|-F file] [-S socket_path] [-fPRU]  [-l
       secs] [-r rawlog_path] [-m mailbox] [-g mailbox_guid] [-n namespace|-N]
       [-x exclude] [-s	state] [-T secs] [-t start date] [-e end date] -d|des-
       tination

DESCRIPTION
       dsync is	Dovecot's mailbox synchronization utility.  It can be used for
       several different use cases: Two-way synchronization of mailboxes, cre-
       ating backups of	mails, and convert mailboxes from/to different mailbox
       formats.	 All of	these can be used within the same  server  or  between
       different  servers  (via	 ssh(1)	or tcp connections).  Remote mailboxes
       can be accessed also via	IMAP protocol, which allows  using  dsync  for
       mailbox migration purposes.

       You can run dsync in one	of three modes:

	      o	     doveadm  backup  performs	one-way	 synchronization.   If
		     there are any changes in the  destination	they  will  be
		     deleted,  so  the	destination will look exactly like the
		     source.

	      o	     doveadm sync performs two-way synchronization.  It	merges
		     all  changes without losing anything.  Both the mailboxes
		     will end up looking identical after  the  synchronization
		     is	finished.

	      o	     doveadm  sync -1 performs one-way synchronization,	but it
		     merges the	changes	in destination without	deleting  any-
		     thing.  This doesn't currently work perfectly, so its use
		     should be limited.	 Its main purpose is that during mail-
		     box  migration you	can run	doveadm	backup multiple	times,
		     then switch mails to be delivered to the new mailbox  and
		     run  doveadm  sync	 -1 once more to transfer any last new
		     mails from	the old	mailbox.

		     The one-way algorithm is the same as two-way dsync	 algo-
		     rithm  except  the	 source	 account  is  not modified. It
		     fetches the message's GUID	(Global	UID), which is used to
		     identify any conflicting UIDs in messages.	As long	as the
		     source and	destination side has matching UID<->GUID  map-
		     ping,  those  emails  are assumed to be synced correctly.
		     Only after	the first mismatch will	changes	begin.

		     Example: Source mailbox has  messages  UID	 1..5;	source
		     mailbox  is  sync'd  using	doveadm	backup to the destina-
		     tion. Subsequently, UID 6	is  delivered  to  the	source
		     mailbox  and UID 1	is expunged from the destination mail-
		     box. In this example, UID 1 is kept removed (in  destina-
		     tion)  because  UID  1..5	have identical Date+Message-ID
		     headers. UID 6 is not seen	in destination so it's copied.

		     If	both source and	destination have UID 6,	but  the  mes-
		     sages are different, the headers don't match and both the
		     messages are kept in the destination  but	they're	 given
		     new  UIDs	7  and 8 just to be sure any client didn't get
		     confused about what UID 6	actually  was.	Thus,  one-way
		     sync  begins  to  quickly diverge from the	source mailbox
		     once changes start	to occur on either side; one-way  sync
		     should therefore normally only be used within a short pe-
		     riod of time after	a doveadm backup or doveadm sync  com-
		     mand was used to synchronize the mailboxes.

       There are also three different synchronization algorithms:

	      o	     Full synchronization (-f parameter) scans through all the
		     messages in all the mailboxes.  This guarantees that  ev-
		     erything  will  be	 synchronized,	but it's unnecessarily
		     slow for incremental synchronization.

	      o	     Fast synchronization (default)  first  attempts  to  find
		     mailboxes	that have changed, and synchronize only	those.
		     This is done by checking the mailboxes' metadata (NEXTUID
		     and  HIGHESTMODSEQ).  Usually this	works fine, especially
		     with one-way synchronization, but if both	sides  do  ex-
		     actly the same number of changes, the metadata may	end up
		     containing	the same values	even if	the changes were  dif-
		     ferent.

	      o	     Stateful synchronization (-s parameter) is	the most effi-
		     cient way to synchronize mailboxes.  It relies on	having
		     the  earlier  dsync run's state saved somewhere and being
		     passed to the next	dsync run.  Based on this state	 dsync
		     can  send only the	changes	that happened after the	previ-
		     ous dsync run.  As	long as	the  state  or	the  mailboxes
		     aren't  corrupted	this  algorithm	should work perfectly.
		     The replicator process uses this  internally  to  perform
		     most of the synchronization.

       The  syncing is done as perfectly as possible: an IMAP or a POP3	client
       shouldn't be able to notice any differences between the two  mailboxes.
       Two-way syncing means that it's safe to do any kind of modifications in
       both sides, and dsync will merge	the changes without losing any changes
       done  on	 either	side.  This is possible	because	dsync can access Dove-
       cot's index logs	that keep track	of changes.  It's of  course  possible
       to  have	 conflicts  during  merging, these are resolved	in a safe way.
       See the dsync design document for more information.

       dsync uses the same configuration files as the  rest  of	 Dovecot  (via
       doveconf(1) binary).  The entire	configuration can be changed by	giving
       -c parameter to another configuration file, or using  -o	 parameter  to
       override	 specific  settings.  When executing a remote dsync program it
       works the same way: it uses its own local configuration.

       dsync can be run	completely standalone.	It doesn't require any Dovecot
       server  processes to be running,	except when using -u parameter to do a
       userdb lookup from auth process.

       dsync can sync either one or multiple users using the -u	or -A  parame-
       ters.   For  continuous	replication you	can use	the Dovecot replicator
       process,	which automatically runs dsync whenever	messages have changed.

OPTIONS
       Global doveadm(1) options:

       -D     Enables verbosity	and debug messages.

       -o setting=value
	      Overrides	the configuration  setting  from  /usr/local/etc/dove-
	      cot/dovecot.conf	and  from the userdb with the given value.  In
	      order to override	multiple settings, the -o option may be	speci-
	      fied multiple times.

       -v     Enables verbosity, including progress counter.

       Command specific	options:

       -1     Do one-way synchronization instead of two-way synchronization.

       -A     If  the  -A option is present, the command will be performed for
	      all users.  Using	this option in combination with	 system	 users
	      from  userdb  { driver = passwd }	is not recommended, because it
	      contains also users with a lower UID  than  the  one  configured
	      with the first_valid_uid setting.

	      When  the	 SQL  userdb  module  is used make sure	that the iter-
	      ate_query	setting	in /usr/local/etc/dovecot/dovecot-sql.conf.ext
	      matches  your  database layout.  When using the LDAP userdb mod-
	      ule, make	sure that the iterate_attrs  and  iterate_filter  set-
	      tings in /usr/local/etc/dovecot/dovecot-ldap.conf.ext match your
	      LDAP schema.  Otherwise doveadm(1) will  be  unable  to  iterate
	      over all users.

       -F file
	      Execute the command for all the users in the file.  This is sim-
	      ilar to the -A option, but instead of getting the	list of	 users
	      from  the	 userdb,  they are read	from the given file.  The file
	      contains one username per	line.

       -N     Synchronize all the available namespaces.	 By default only name-
	      spaces  that  don't  have	explicit location setting are synchro-
	      nized.

       -P     Run a doveadm-purge(1) for the destination (remote) storage  af-
	      ter synchronization.

       -R     Do  a  reverse sync. Normally, messages would be pushed from the
	      local system to the destination (remote).	This  option  reverses
	      the  flow, and will instead pull messages	from the remote	to the
	      local storage.

       -S socket_path
	      The option's argument is either an absolute path to a local UNIX
	      domain  socket, or a hostname and	port (hostname:port), in order
	      to connect a remote host via a TCP socket.

	      This allows an administrator to execute doveadm(1) mail commands
	      through the given	socket.

       -T secs
	      Specify the time in seconds, how long doveadm(1) should wait for
	      stalled I/O operations.  The default timeout is 600 seconds.

       -U     This is used internally by replicator to have  dsync  notify  it
	      when the synchronization is finished.

       -d     Use  the	default	 destination,  which  is  looked  up  from the
	      mail_replica userdb extra	field.

       -g mailbox_guid
	      Same as -m, but find the mailbox to be synchronized by its  GUID
	      instead of by name.

       -l secs
	      Lock the dsync for this user.  Wait for maximum secs before giv-
	      ing up.  This parameter should be	used to	avoid broken  synchro-
	      nization	if  it's possible that dsync is	being run concurrently
	      for the same user.

       -m mailbox
	      Synchronize only this mailbox name.

       -n namespace
	      Synchronize only the specified namespace.	 This parameter	can be
	      used multiple times.

       -r rawlog_path
	      Running dsync remotely, write the	remote input/output traffic to
	      the specified log	file.

       -s previous_state
	      Use stateful synchronization.  If	the previous state is unknown,
	      use  an  empty string.  The new state is always printed to stan-
	      dard output.

       -u user/mask
	      Run the command only for the given user.	It's also possible  to
	      use '*' and '?' wildcards	(e.g. -u *@example.org).
	      When  neither  the  -A  option,  nor the -F file option, nor the
	      -u user was specified, the command will be executed with the en-
	      vironment	of the currently logged	in user.

       -x mailbox_mask
	      Exclude  the  specified mailbox name/mask.  The mask may contain
	      "?" and "*" wildcards.  This  parameter  can  be	used  multiple
	      times.

ARGUMENTS
       destination
	      This argument specifies the synchronized destination.  It	can be
	      one of:

	      location
		     Same as mail_location setting, e.g. maildir:~/Maildir

	      remote:login@host
		     Uses dsync_remote_cmd setting to connect  to  the	remote
		     host (usually via ssh)

	      remoteprefix:login@host
		     This  is  the  same  as remote, except "user@domain\n" is
		     sent before dsync protocol	starts.	  This	allows	imple-
		     menting  a	 trusted  wrapper  script  that	 runs  doveadm
		     dsync-server by reading the username from the first line.

	      tcp:host[:port]
		     Connects to remote	doveadm	server via TCP.	  The  default
		     port is specified by doveadm_port setting.

	      tcps:host[:port]
		     This is the same as tcp, but with SSL.

EXIT STATUS
       dsync will exit with one	of the following values:

       0   Synchronization was done perfectly.

       2   Synchronization  was	done without errors, but some changes couldn't
	   be done, so the mailboxes aren't  perfectly	synchronized.  Running
	   dsync  again	 usually fixes this. Typically this occurs for message
	   modification	sequences with newly created mailboxes.	 It  can  also
	   occur if one	of the mailboxes change	during the syncing.

       1, >2
	   Synchronization failed.

EXAMPLE
   SYNCHRONIZATION
       Synchronize  mailboxes with a remote server.  Any errors	are written to
       stderr.

	      doveadm sync -u username@example.com remote:server-replica.example.com

       If you need more	complex	parameters to ssh, you can use e.g.:

	      doveadm sync -u username@example.com ssh -i id_dsa.dovecot \
	      mailuser@example.com doveadm dsync-server	-u username@example.com

   CONVERTING
       Assuming	 that  the  mail_location  setting   in	  /usr/local/etc/dove-
       cot/conf.d/10-mail.conf	is  set	 to:  mail_location = mdbox:~/mdbox, a
       logged in system	user may convert her/his mails	from  its  Maildir  in
       her/his	home  directory	 to the	mdbox mailbox format.  The user	has to
       execute the command:

	      doveadm sync maildir:~/Maildir

       If you want to do this without any downtime, you	can do the  conversion
       one user	at a time.  Initially:

	   o   Configuration uses mail_location	= maildir:~/Maildir

	   o   Set  up	the  possibility of doing per-user mail	location using
	       userdb extra fields.

       Then for	each user:

	   1.  Run doveadm sync	once to	do the initial conversion.

	   2.  Run doveadm sync	again, because the  initial  conversion	 could
	       have  taken  a while and	new changes could have occurred	during
	       it.  This second	time only applies changes,  so	it  should  be
	       fast.

	   3.  Update  mail extra field	in userdb to mdbox:~/mdbox.  If	you're
	       using auth cache, you need to  flush  it,  e.g.	 doveadm  auth
	       cache flush.

	   4.  Wait  for a few seconds and then	kill (doveadm kick) the	user's
	       all existing imap and  pop3  sessions  (that  are  still	 using
	       maildir).

	   5.  Run  doveadm  sync  once	 more to apply final changes that were
	       possibly	done.  After  this  there  should  be  no  changes  to
	       Maildir,	 because the user's mail location has been changed and
	       all existing processes using it have been killed.

       Once all	users have been	converted, you can set the default  mail_loca-
       tion to mdbox and remove	the per-user mail locations from userdb.

REPORTING BUGS
       Report  bugs, including doveconf	-n output, to the Dovecot Mailing List
       <dovecot@dovecot.org>.  Information about reporting bugs	 is  available
       at: http://dovecot.org/bugreport.html

SEE ALSO
       doveadm(1),  doveadm-auth(1),  doveadm-kick(1), doveadm-purge(1), dove-
       conf(1)

       Additional resources:

       dsync design
	      http://wiki2.dovecot.org/Design/Dsync

Dovecot	v2.3			  2015-08-31		       DOVEADM-SYNC(1)

NAME | SYNOPSIS | DESCRIPTION | OPTIONS | ARGUMENTS | EXIT STATUS | EXAMPLE | REPORTING BUGS | SEE ALSO

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=dsync&sektion=1&manpath=FreeBSD+13.0-RELEASE+and+Ports>

home | help