Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages


home | help
FPSYNC(1)		  BSD General Commands Manual		     FPSYNC(1)

     fpsync -- Synchronize directories in parallel using fpart and rsync.

     fpsync [-h] [-v] [-n jobs]	[-f files] [-s size] [-w wrks] [-d shdir]
	    [-t	tmpdir]	[-r jobname] [-o rsyncopts] [-O	fpartopts] [-S]
	    src_dir/ dst_url/

     The fpsync	tool synchronizes directories in parallel using	fpart(1) and
     rsync(1).	It computes subsets of src_dir/	and spawns rsync(1) jobs to
     synchronize them to dst_url/.

     Synchronization jobs can be executed either locally or remotely (using
     SSH workers, see option -w) and are executed on-the-fly while filesystem
     crawling goes on. This makes fpsync a good	tool for migrating large

     -h	     Print help

     -v	     Verbose mode. Can be be specified several times to	increase ver-
	     bosity level.

     -n	jobs
	     Start jobs	concurrent sync	jobs (either locally or	remotely, see
	     below). Default: 2

     -f	files
	     Transfer at most files files per sync job.	Default: 2000

     -s	size
	     Transfer at most size bytes per sync job.
	     Default: 4294967296 (4 GB)

     -w	wrks
	     Use remote	SSH wrks to synchronize	files. Synchronization jobs
	     are executed locally when this option is not set.	wrks is	a
	     space-separated list of login@machine connection strings and can
	     be	specified several times. You must be allowed to	connect	to
	     those machines using a SSH	key to avoid user interaction.

     -d	shdir
	     Set fpsync	shared directory to shdir.  This option	is mandatory
	     when using	SSH workers and	set by default to tmpdir when running
	     locally. The specified directory must be an absolute path ; it
	     will be used to handle communications with	SSH hosts (sharing
	     partitions	and log	files) and, as a consequence, must be made
	     available to all participating hosts (e.g.	through	a r/w NFS
	     mount), including the master one running fpsync.

     -t	tmpdir
	     Set fpsync	temporary directory to tmpdir.	This directory remains
	     local and does not	need to	be shared amongst SSH workers when us-
	     ing the -w	option.	Default: /tmp/fpsync

     -r	jobname
	     Resume job	jobname	and restart synchronizing remaining partitions
	     from a previous run.  jobname can be obtained using verbose mode
	     (see option -v).  Note that filesystem crawling is	skipped	when
	     resuming a	previous run. As a consequence,	options	-f, -s,	-o,
	     -O, -S, src_dir/, and dst_url/ are	ignored.

     -o	rsyncopts
	     Override default rsync(1) options with rsyncopts.	Use this op-
	     tion with care as certain options are incompatible	with a paral-
	     lel usage (e.g.  --delete).  Default: -av --numeric-ids

     -O	fpartopts
	     Override default fpart(1) options with fpartopts.
	     Default: -x .zfs -x .snapshot* -x .ckpt

     -S	     Sudo mode.	Use sudo(8) for	filesystem crawling and	synchroniza-

	     Source directory. It must be absolute and available on all	par-
	     ticipating	hosts (including the master one, running fpsync).

	     Destination URL. It may be	a local	path or	a remote URL, as sup-
	     ported by rsync(1).  All participating workers must be able to
	     reach that	target.

     Each fpsync run generates a unique	jobname, which is displayed in verbose
     mode (see option -v) and within log files.	 You can use that jobname to
     resume a previous run (see	option -r).  fpsync will then restart synchro-
     nizing data from the parts	that were being	synchonized at the time	it

     This unique feature gives the administrator the ability to	stop fpsync
     and restart it later, without having to restart the whole filesystem
     crawling and synchronization process. Note	that resuming is only possible
     when filesystem crawling step has finished.

     During synchronization, you can press CTRL-C to interrupt the process.
     The first CTRL-C prevents new synchronizations from being submitted and
     the process will wait for current synchronizations	to be finished before
     exiting.  If you press CTRL-C again, current synchronizations will	be
     killed and	fpsync will exit immediately.

     On	certain	systems, CTRL-T	can be pressed to get the status of current
     and remaining parts to be synchronized. This can also be achieved by
     sending a SIGINFO to the fpsync process.

     Whether you use verbose mode or not, everything is	logged within

     Here are some examples:

     fpsync -n 4 /usr/src/ /var/src/

	     Synchronizes /usr/src/ to /var/src/ using 4 local jobs.

     fpsync -n 2 -w login@machine1 -w login@machine2 -d	/mnt/fpsync /mnt/src/

	     Synchronizes /mnt/src/ to /mnt/dst/ using 2 concurrent jobs exe-
	     cuted remotely on 2 SSH workers (machine1 and machine2). The
	     shared directory is set to	/mnt/fpsync and	mounted	on the machine
	     running fpsync, as	well as	on machine1 and	machine2. The source
	     directory (/mnt/src/) is also available on	those 3	machines,
	     while the destination directory (/mnt/dst/) is mounted on SSH
	     workers only (machine1 and	machine2).

     Parallelizing rsync(1) makes several options not usable, such as
     --delete.	If your	source directory is live while fpsync is running, you
     will have to delete extra files from destination directory. This is usu-
     ally done by using	a final	-offline- rsync(1) pass	that will use this op-

     fpsync enqueues synchronization jobs on disk, within the tmpdir/queue di-
     rectory. Be careful to host this queue on a filesystem that can handle
     fine-grained mtime	timestamps (i.e. with a	sub-second precision) if you
     want the queue to be processed in order when fpart(1) generates several
     jobs per second. On FreeBSD, VFS(9) timestamps' precision can be tuned
     using the 'vfs.timestamp_precision' sysctl. See vfs_timestamp(9).

     fpart(1), rsync(1), sudo(8)

     Fpsync has	been written by	Ganael LAPLANCHE and is	available under	the
     BSD license on

     No	bug known (yet).

BSD			       January 27, 2015				   BSD


Want to link to this manual page? Use this URL:

home | help