Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages


home | help
RBD(8)				     Ceph				RBD(8)

       rbd - manage rados block	device (RBD) images

       rbd [ -c	ceph.conf ] [ -m monaddr ] [--cluster cluster-name]
       [ -p | --pool pool ] [ command ... ]

       rbd is a	utility	for manipulating rados block device (RBD) images, used
       by the Linux rbd	driver and the rbd storage driver for  QEMU/KVM.   RBD
       images  are  simple  block  devices  that  are striped over objects and
       stored in a RADOS object	store. The size	of the objects	the  image  is
       striped over must be a power of two.

       -c ceph.conf, --conf ceph.conf
	      Use   ceph.conf	configuration  file  instead  of  the  default
	      /etc/ceph/ceph.conf  to  determine  monitor   addresses	during

       -m monaddress[:port]
	      Connect	to  specified  monitor	(instead  of  looking  through

       --cluster cluster-name
	      Use different cluster name as compared to	default	 cluster  name

       -p pool-name, --pool pool-name
	      Interact with the	given pool. Required by	most commands.

	      Do  not  output  progress	information (goes to standard error by
	      default for some commands).

       --image-format format-id
	      Specifies	which object layout to use. The	default	is 2.

	      o	format 1 - (deprecated)	Use the	original format	for a new  rbd
		image. This format is understood by all	versions of librbd and
		the kernel rbd module, but does	 not  support  newer  features
		like cloning.

	      o	format	2  -  Use the second rbd format, which is supported by
		librbd and kernel since	version	3.11  (except  for  striping).
		This adds support for cloning and is more easily extensible to
		allow more features in the future.

       -s size-in-M/G/T, --size	size-in-M/G/T
	      Specifies	the size of the	new rbd	image or the new size  of  the
	      existing	rbd  image in M/G/T.  If no suffix is given, unit M is

       --object-size size-in-B/K/M
	      Specifies	the object size	in B/K/M.  Object size will be rounded
	      up  the  nearest	power of two; if no suffix is given, unit B is
	      assumed.	The default object size	is 4M, smallest	is 4K and max-
	      imum is 32M.

       --stripe-unit size-in-B/K/M
	      Specifies	the stripe unit	size in	B/K/M.	If no suffix is	given,
	      unit B is	assumed.  See striping section (below)	for  more  de-

       --stripe-count num
	      Specifies	 the  number  of objects to stripe over	before looping
	      back to the first	object.	 See striping section (below) for more

       --snap snap
	      Specifies	the snapshot name for the specific operation.

       --id username
	      Specifies	 the username (without the client. prefix) to use with
	      the map command.

       --keyring filename
	      Specifies	a keyring file containing a secret for	the  specified
	      user to use with the map command.	 If not	specified, the default
	      keyring locations	will be	searched.

       --keyfile filename
	      Specifies	a file containing the secret key of --id user  to  use
	      with the map command.  This option is overridden by --keyring if
	      the latter is also specified.

       --shared	lock-tag
	      Option for lock add that allows multiple	clients	 to  lock  the
	      same  image  if  they  use the same tag. The tag is an arbitrary
	      string. This is useful for situations where  an  image  must  be
	      open  from more than one client at once, like during live	migra-
	      tion of a	virtual	machine, or for	 use  underneath  a  clustered

       --format	format
	      Specifies	output formatting (default: plain, json, xml)

	      Make json	or xml formatted output	more human-readable.

       -o krbd-options,	--options krbd-options
	      Specifies	 which options to use when mapping or unmapping	an im-
	      age via the rbd kernel driver.  krbd-options  is	a  comma-sepa-
	      rated  list of options (similar to mount(8) mount	options).  See
	      kernel rbd (krbd)	options	section	below for more details.

	      Map the image read-only.	Equivalent to -o ro.

       --image-feature feature-name
	      Specifies	which RBD format 2 feature should be enabled when cre-
	      ating  an	 image.	 Multiple features can be enabled by repeating
	      this option multiple times.  The	following  features  are  sup-

	      o	layering: layering support

	      o	striping: striping v2 support

	      o	exclusive-lock:	exclusive locking support

	      o	object-map: object map support (requires exclusive-lock)

	      o	fast-diff: fast	diff calculations (requires object-map)

	      o	deep-flatten: snapshot flatten support

	      o	journaling: journaled IO support (requires exclusive-lock)

	      Specifies	 that  the image will be used concurrently by multiple
	      clients.	This will disable features that	are dependent upon ex-
	      clusive ownership	of the image.

	      Specifies	 that  the  diff should	be limited to the extents of a
	      full object instead of showing intra-object deltas. When the ob-
	      ject  map	 feature  is enabled on	an image, limiting the diff to
	      the object extents will dramatically improve  performance	 since
	      the  differences	can be computed	by examining the in-memory ob-
	      ject map instead of querying RADOS for each  object  within  the

	      Specifies	the limit for the number of snapshots permitted.

       ls [-l |	--long]	[pool-name]
	      Will  list  all  rbd  images listed in the rbd_directory object.
	      With -l, also show snapshots, and	use longer-format  output  in-
	      cluding size, parent (if clone), format, etc.

       du [-p |	--pool pool-name] [image-spec |	snap-spec]
	      Will  calculate the provisioned and actual disk usage of all im-
	      ages and associated snapshots within the specified pool.	It can
	      also be used against individual images and snapshots.

	      If the RBD fast-diff feature isn't enabled on images, this oper-
	      ation will require querying the OSDs for every potential	object
	      within the image.

       info image-spec | snap-spec
	      Will  dump  information  (such  as size and object size) about a
	      specific rbd image.  If image is a clone,	information about  its
	      parent  is  also displayed.  If a	snapshot is specified, whether
	      it is protected is shown as well.

       create (-s | --size size-in-M/G/T)  [--image-format  format-id]	[--ob-
       ject-size  size-in-B/K/M]  [--stripe-unit  size-in-B/K/M	--stripe-count
       num] [--image-feature feature-name]... [--image-shared] image-spec
	      Will create a new	rbd image. You must also specify the size  via
	      --size.	The --stripe-unit and --stripe-count arguments are op-
	      tional, but must be used together.

       clone  [--object-size   size-in-B/K/M]	[--stripe-unit	 size-in-B/K/M
       --stripe-count  num]  [--image-feature  feature-name]  [--image-shared]
       parent-snap-spec	child-image-spec
	      Will create a clone (copy-on-write child)	of  the	 parent	 snap-
	      shot.  Object size will be identical to that of the parent image
	      unless specified.	Size will be the same as the parent  snapshot.
	      The --stripe-unit	and --stripe-count arguments are optional, but
	      must be used together.

	      The parent snapshot must be protected (see  rbd  snap  protect).
	      This requires image format 2.

       flatten image-spec
	      If  image	 is  a	clone,	copy all shared	blocks from the	parent
	      snapshot and make	the child independent of the parent,  severing
	      the link between parent snap and child.  The parent snapshot can
	      be unprotected and  deleted  if  it  has	no  further  dependent

	      This requires image format 2.

       children	snap-spec
	      List  the	clones of the image at the given snapshot. This	checks
	      every pool, and outputs the resulting poolname/imagename.

	      This requires image format 2.

       resize (-s | --size size-in-M/G/T) [--allow-shrink] image-spec
	      Resizes rbd image. The size parameter also needs	to  be	speci-
	      fied.  The --allow-shrink	option lets the	size be	reduced.

       rm image-spec
	      Deletes  an  rbd image (including	all data blocks). If the image
	      has snapshots, this fails	and nothing is deleted.

       export [--export-format format  (1  or  2)]  (image-spec	 |  snap-spec)
	      Exports  image  to  dest	path  (use  -  for stdout).  The --ex-
	      port-format accepts '1' or '2' currently.	Format 2 allow	us  to
	      export not only the content of image, but	also the snapshots and
	      other properties,	such as	image_order, features.

       import [--export-format format (1  or  2)]  [--image-format  format-id]
       [--object-size	   size-in-B/K/M]     [--stripe-unit	 size-in-B/K/M
       --stripe-count num] [--image-feature feature-name]...  [--image-shared]
       src-path	[image-spec]
	      Creates  a  new  image and imports its data from path (use - for
	      stdin).  The import operation will try to	create sparse rbd  im-
	      ages  if	possible.   For	 import	from stdin, the	sparsification
	      unit is the data block size of  the  destination	image  (object

	      The --stripe-unit	and --stripe-count arguments are optional, but
	      must be used together.

	      The --export-format accepts '1' or '2' currently.	Format 2 allow
	      us  to  import not only the content of image, but	also the snap-
	      shots and	other properties, such as image_order, features.

       export-diff  [--from-snap  snap-name]  [--whole-object]	(image-spec  |
       snap-spec) dest-path
	      Exports an incremental diff for an image to dest path (use - for
	      stdout).	If an initial  snapshot	 is  specified,	 only  changes
	      since  that snapshot are included; otherwise, any	regions	of the
	      image that contain data are included.  The end snapshot is spec-
	      ified  using the standard	--snap option or @snap syntax (see be-
	      low).  The image diff format includes metadata about image  size
	      changes, and the start and end snapshots.	 It efficiently	repre-
	      sents discarded or 'zero'	regions	of the image.

       merge-diff first-diff-path second-diff-path merged-diff-path
	      Merge two	continuous incremental diffs of	an image into one sin-
	      gle  diff.  The first diff's end snapshot	must be	equal with the
	      second diff's start snapshot.  The first diff  could  be	-  for
	      stdin, and merged	diff could be -	for stdout, which enables mul-
	      tiple  diff  files  to  be  merged  using	 something  like  'rbd
	      merge-diff first second -	| rbd merge-diff - third result'. Note
	      this command currently only support the source incremental  diff
	      with stripe_count	== 1

       import-diff src-path image-spec
	      Imports  an  incremental	diff of	an image and applies it	to the
	      current image.  If the diff was generated	relative  to  a	 start
	      snapshot,	we verify that snapshot	already	exists before continu-
	      ing.  If there was an end	snapshot we verify it does not already
	      exist  before applying the changes, and create the snapshot when
	      we are done.

       diff [--from-snap snap-name] [--whole-object] image-spec	| snap-spec
	      Dump a list of byte extents in the image that have changed since
	      the  specified  start  snapshot, or since	the image was created.
	      Each output line includes	the starting offset  (in  bytes),  the
	      length  of the region (in	bytes),	and either 'zero' or 'data' to
	      indicate whether the region is known to be zeros or may  contain
	      other data.

       cp (src-image-spec | src-snap-spec) dest-image-spec
	      Copies  the  content  of	a  src-image  into  the	 newly created
	      dest-image.  dest-image will have	the same  size,	 object	 size,
	      and image	format as src-image.

       mv src-image-spec dest-image-spec
	      Renames an image.	 Note: rename across pools is not supported.

       image-meta list image-spec
	      Show metadata held on the	image. The first column	is the key and
	      the second column	is the value.

       image-meta get image-spec key
	      Get metadata value with the key.

       image-meta set image-spec key value
	      Set metadata key with the	value.	They  will  displayed  in  im-
	      age-meta list.

       image-meta remove image-spec key
	      Remove metadata key with the value.

       object-map rebuild image-spec | snap-spec
	      Rebuilds an invalid object map for the specified image. An image
	      snapshot can be specified	to rebuild an invalid object map for a

       snap ls image-spec
	      Dumps the	list of	snapshots inside a specific image.

       snap create snap-spec
	      Creates  a  new  snapshot.  Requires the snapshot	name parameter

       snap rollback snap-spec
	      Rollback image content to	snapshot. This	will  iterate  through
	      the  entire blocks array and update the data head	content	to the
	      snapshotted version.

       snap rm [--force] snap-spec
	      Removes the specified snapshot.

       snap purge image-spec
	      Removes all snapshots from an image.

       snap protect snap-spec
	      Protect a	snapshot from deletion,	so that	clones can be made  of
	      it  (see	rbd clone).  Snapshots must be protected before	clones
	      are made;	protection implies that	there exist  dependent	cloned
	      children	that refer to this snapshot.  rbd clone	will fail on a
	      nonprotected snapshot.

	      This requires image format 2.

       snap unprotect snap-spec
	      Unprotect	a snapshot from	 deletion  (undo  snap	protect).   If
	      cloned children remain, snap unprotect fails.  (Note that	clones
	      may exist	in different pools than	the parent snapshot.)

	      This requires image format 2.

       snap limit set [--limit]	limit image-spec
	      Set a limit for the number of snapshots allowed on an image.

       snap limit clear	image-spec
	      Remove any previously set	limit on the number of	snapshots  al-
	      lowed on an image.

       map [-o | --options krbd-options	] [--read-only]	image-spec | snap-spec
	      Maps  the	 specified  image to a block device via	the rbd	kernel

       unmap [-o | --options krbd-options  ]  image-spec  |  snap-spec	|  de-
	      Unmaps  the block	device that was	mapped via the rbd kernel mod-

	      Show the rbd images that are mapped via the rbd kernel module.

       nbd map [--device device-path] [--read-only] image-spec | snap-spec
	      Maps the specified image to a block device via the rbd-nbd tool.

       nbd unmap device-path
	      Unmaps the block device that was mapped via the rbd-nbd tool.

       nbd list
	      Show the list of used nbd	devices	via the	rbd-nbd	tool.

       status image-spec
	      Show the status of the image, including which  clients  have  it

       feature disable image-spec feature-name...
	      Disables	the specified feature on the specified image. Multiple
	      features can be specified.

       feature enable image-spec feature-name...
	      Enables the specified feature on the specified  image.  Multiple
	      features can be specified.

       lock list image-spec
	      Show  locks held on the image. The first column is the locker to
	      use with the lock	remove command.

       lock add	[--shared lock-tag] image-spec lock-id
	      Lock an image. The lock-id is an arbitrary name for  the	user's
	      convenience.  By	default, this is an exclusive lock, meaning it
	      will fail	if the image is	already	locked.	 The  --shared	option
	      changes this behavior. Note that locking does not	affect any op-
	      eration other than adding	a lock.	It does	not protect  an	 image
	      from being deleted.

       lock remove image-spec lock-id locker
	      Release a	lock on	an image. The lock id and locker are as	output
	      by lock ls.

       bench  --io-type	  <read	  |   write>   [--io-size   size-in-B/K/M/G/T]
       [--io-threads	num-ios-in-flight]    [--io-total   size-in-B/K/M/G/T]
       [--io-pattern seq | rand] image-spec
	      Generate a series	of  IOs	 to  the  image	 and  measure  the  IO
	      throughput  and  latency.	  If no	suffix is given, unit B	is as-
	      sumed  for  both	--io-size  and	--io-total.    Defaults	  are:
	      --io-size	 4096,	--io-threads  16,  --io-total 1G, --io-pattern

       image-spec is [pool-name]/image-name
       snap-spec  is [pool-name]/image-name@snap-name

       The default for pool-name is "rbd".  If an image	name contains a	 slash
       character ('/'),	pool-name is required.

       You  may	 specify  each	name  individually,  using --pool, --image and
       --snap options, but this	is discouraged in favor	of the above spec syn-

       RBD  images are striped over many objects, which	are then stored	by the
       Ceph distributed	object store (RADOS).  As a result, read and write re-
       quests  for the image are distributed across many nodes in the cluster,
       generally preventing any	single node from becoming  a  bottleneck  when
       individual images get large or busy.

       The striping is controlled by three parameters:

	      The size of objects we stripe over is a power of two. It will be
	      rounded up the nearest power of two.  The	default	object size is
	      4	MB, smallest is	4K and maximum is 32M.

	      Each [stripe_unit] contiguous bytes are stored adjacently	in the
	      same object, before we move on to	the next object.

	      After we write [stripe_unit] bytes to [stripe_count] objects, we
	      loop  back to the	initial	object and write another stripe, until
	      the object reaches its maximum size.  At that point, we move  on
	      to the next [stripe_count] objects.

       By   default,  [stripe_unit]  is	 the  same  as	the  object  size  and
       [stripe_count] is 1.  Specifying	 a  different  [stripe_unit]  requires
       that the	STRIPINGV2 feature be supported	(added in Ceph v0.53) and for-
       mat 2 images be used.

       Most of these options are useful	mainly for debugging and benchmarking.
       The  default  values  are set in	the kernel and may therefore depend on
       the version of the running kernel.

       Per client instance rbd map options:

       o fsid=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee - FSID that should  be  as-
	 sumed by the client.

       o ip=a.b.c.d[:p]	- IP and, optionally, port the client should use.

       o share	-  Enable sharing of client instances with other mappings (de-

       o noshare - Disable sharing of client instances with other mappings.

       o crc - Enable CRC32C checksumming for data writes (default).

       o nocrc - Disable CRC32C	checksumming for data writes.

       o cephx_require_signatures - Require cephx message signing (since 3.19,

       o nocephx_require_signatures  -	Don't  require	cephx  message signing
	 (since	3.19).

       o tcp_nodelay - Disable Nagle's algorithm on client sockets (since 4.0,

       o notcp_nodelay	-  Enable  Nagle's  algorithm on client	sockets	(since

       o cephx_sign_messages - Enable message signing (since 4.4, default).

       o nocephx_sign_messages - Disable message signing (since	4.4).

       o mount_timeout=x - A timeout on	various	steps in rbd map and rbd unmap
	 sequences (default is 60 seconds).  In	particular, since 4.2 this can
	 be used to ensure that	rbd unmap eventually times out when  there  is
	 no network connection to a cluster.

       o osdkeepalive=x	- OSD keepalive	timeout	(default is 5 seconds).

       o osd_idle_ttl=x	- OSD idle TTL (default	is 60 seconds).

       Per mapping (block device) rbd map options:

       o rw - Map the image read-write (default).

       o ro - Map the image read-only.	Equivalent to --read-only.

       o queue_depth=x - queue depth (since 4.2, default is 128	requests).

       o lock_on_read -	Acquire	exclusive lock on reads, in addition to	writes
	 and discards (since 4.9).

       o exclusive -  Disable  automatic  exclusive  lock  transitions	(since

       rbd unmap options:

       o force	-  Force  the  unmapping of a block device that	is open	(since
	 4.9).	The driver will	wait for running requests to complete and then
	 unmap;	requests sent to the driver after initiating the unmap will be

       To create a new rbd image that is 100 GB:

	  rbd create mypool/myimage --size 102400

       To use a	non-default object size	(8 MB):

	  rbd create mypool/myimage --size 102400 --object-size	8M

       To delete an rbd	image (be careful!):

	  rbd rm mypool/myimage

       To create a new snapshot:

	  rbd snap create mypool/myimage@mysnap

       To create a copy-on-write clone of a protected snapshot:

	  rbd clone mypool/myimage@mysnap otherpool/cloneimage

       To see which clones of a	snapshot exist:

	  rbd children mypool/myimage@mysnap

       To delete a snapshot:

	  rbd snap rm mypool/myimage@mysnap

       To map an image via the kernel with cephx enabled:

	  rbd map mypool/myimage --id admin --keyfile secretfile

       To map an image via the kernel with different cluster name  other  than
       default ceph:

	  rbd map mypool/myimage --cluster cluster-name

       To unmap	an image:

	  rbd unmap /dev/rbd0

       To create an image and a	clone from it:

	  rbd import --image-format 2 image mypool/parent
	  rbd snap create mypool/parent@snap
	  rbd snap protect mypool/parent@snap
	  rbd clone mypool/parent@snap otherpool/child

       To  create  an  image  with a smaller stripe_unit (to better distribute
       small writes in some workloads):

	  rbd create mypool/myimage --size 102400 --stripe-unit	65536B --stripe-count 16

       To change an image from one image format	to another, export it and then
       import it as the	desired	image format:

	  rbd export mypool/myimage@snap /tmp/img
	  rbd import --image-format 2 /tmp/img mypool/myimage2

       To lock an image	for exclusive use:

	  rbd lock add mypool/myimage mylockid

       To release a lock:

	  rbd lock remove mypool/myimage mylockid client.2485

       rbd  is	part  of  Ceph,	a massively scalable, open-source, distributed
       storage	system.	 Please	  refer	  to   the   Ceph   documentation   at for	more information.

       ceph(8),	rados(8)

       2010-2014,  Inktank Storage, Inc. and contributors. Licensed under Cre-
       ative Commons BY-SA

dev				 Jul 08, 2017				RBD(8)


Want to link to this manual page? Use this URL:

home | help