Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
VENTI-FMT(8)		    System Manager's Manual		  VENTI-FMT(8)

NAME
       buildindex,  checkarenas, checkindex, conf, fmtarenas, fmtbloom,	fmtin-
       dex, fmtisect, syncindex	- prepare and maintain a venti server

SYNOPSIS
       venti/fmtarenas [ -4Z ] [ -a arenasize ]	[ -b blocksize ] name file

       venti/fmtisect [	-1Z ] [	-b blocksize ] name file

       venti/fmtbloom [	-n nblocks | -N	nhash ]	[ -s size ] file

       venti/fmtindex [	-a ] venti.conf

       venti/conf [ -w ] partition [ configfile	]

       venti/buildindex	[ -bd ]	[ -i isect ] ... [ -M imemsize ] venti.conf

       venti/checkindex	[ -f ] [ -B blockcachesize ] venti.conf	tmp

       venti/checkarenas [ -afv	] file

DESCRIPTION
       These commands aid in the setup,	maintenance, and  debugging  of	 venti
       servers.	  See  for an overview of the venti system and for an overview
       of the data structures used by the venti	server.

       Note that the units for the various sizes in the	following commands can
       be specified by appending or to indicate	kilobytes, megabytes, or giga-
       bytes respectively.

   Formatting
       To prepare a server for its initial use,	the arena partitions  and  the
       index sections must be formatted	individually, with fmtarenas and fmti-
       sect.  Then the collection of index sections must be  combined  into  a
       venti index with	fmtindex.

       Fmtarenas  formats  the given file, typically a disk partition, into an
       arena partition.	 The arenas in the partition are given	names  of  the
       form  name%d, where %d is replaced with a sequential number starting at
       0.

       Options to fmtarenas are:

       -a arenasize
	      The arenas are of	arenasize bytes.  The default is  512M,	 which
	      was  selected  to	provide	a balance between the number of	arenas
	      and the ability to copy an  arena	 to  external  media  such  as
	      recordable CDs and tapes.

       -b blocksize
	      The  size,  in bytes, for	read and write operations to the file.
	      The size is recorded in the file,	and is	used  by  applications
	      that access the arenas.  The default is 8k.

       -4     Create a `version	4' arena partition for backwards compatibility
	      with old servers.	 The default is	version	5, used	by the current
	      venti server.

       -Z     Do  not zero the data sections of	the arenas.  Using this	option
	      reduces the formatting time but should only be used when	it  is
	      known  that  the file was	already	zeroed.	 (Version 4 only; ver-
	      sion 5 sections are not and do not need to be zeroed.)

       Fmtisect	formats	the given file,	typically a disk partition, as a venti
       index section with the specified	name.  Each of the index sections in a
       venti configuration must	have a unique name.

       Options to fmtisect are:

       -b bucketsize
	      The size of an index bucket, in bytes.  All the  index  sections
	      within  a	 index must have the same bucket size.	The default is
	      8k.

       -1     Create a `version	1' index section for  backwards	 compatibility
	      with old servers.	 The default is	version	2, used	by the current
	      venti server.

       -Z     Do not zero the index.  Using this option	reduces	the formatting
	      time  but	should only be used when it is known that the file was
	      already zeroed.  (Version	1 only;	version	2 sections are not and
	      do not need to be	zeroed.)

       Fmtbloom	formats	the given file as a Bloom filter (see The options are:

       -n nblock | -N nhash
	      The number of blocks expected to be indexed by the filter	or the
	      number of	hash functions to use.	If the -n option is given,  it
	      is  used,	along with the total size of the filter, to compute an
	      appropriate nhash.

       -s size
	      The size of the Bloom filter.  The default is the	total size  of
	      the  file.   In  either case, size is rounded down to a power of
	      two.

       The file	argument in the	commands above can be of the  form  file:lo-hi
       to  specify  a range of the file.  Lo and hi are	specified in bytes but
       can have	the usual k, m,	or g suffixes.	Either lo or hi	may  be	 omit-
       ted.   This notation eliminates the need	to partition raw disks on non-
       Plan 9 systems.

       Fmtindex	reads the configuration	file venti.conf	 and  initializes  the
       index  sections	to form	a usable index structure.  The arena files and
       index sections must have	previously been	formatted using	fmtarenas  and
       fmtisect	respectively.

       The  function  of a venti index is to map a SHA1	fingerprint to a loca-
       tion in the data	section	of one of the arenas.  The index  is  composed
       of blocks, each of which	contains the mapping for a fixed range of pos-
       sible fingerprint values.  Fmtindex determines the mapping between SHA1
       values  and  the	blocks of the collection of index sections.  Once this
       mapping has been	determined, it cannot be  changed  without  rebuilding
       the  index.  The	basic assumption in the	current	implementation is that
       the index structure is sufficiently empty that individual blocks	of the
       index  will  rarely  overflow.	The  total size	of the index should be
       about 2%	to 10% of the total size of the	arenas,	but the	exact percent-
       age  depends  both  on  the index block size and	the compressed size of
       blocks stored.  See the discussion in for more.

       Fmtindex	also computes a	mapping	between	a linear address space and the
       data section of the collection of arenas.  The -a option	can be used to
       add additional arenas to	an index.  To use this feature,	 add  the  new
       arenas  to  venti.conf  after the existing arenas and then run fmtindex
       -a.

       A copy of the above mappings is stored in the header for	 each  of  the
       index sections.	These copies enable buildindex to restore a single in-
       dex section without rebuilding the entire index.

       To make it easier to bootstrap servers, the configuration file  can  be
       stored  in  otherwise  empty space at the beginning of any venti	parti-
       tions using conf.  A partition so branded with a	configuration file can
       be used in place	of a configuration file	when invoking any of the venti
       commands.  By default, conf prints the configuration stored  in	parti-
       tion.   When  invoked with the -w flag, conf reads a configuration file
       from configfile (or else	standard input)	and stores it in partition.

   Checking and	Rebuilding
       Buildindex populates the	 index	for  the  Venti	 system	 described  in
       venti.conf.  The	index must have	previously been	formatted using	fmtin-
       dex.  This command is typically used to build a new index for  a	 Venti
       system when the old index becomes too small, or to rebuild an index af-
       ter media failure.  Small errors	in an index can	usually	be fixed  with
       checkindex,  but	 checkindex  requires  a large temporary workspace and
       buildindex does not.

       Options to buildindex are:

       -b     Reinitialise the Bloom filter, if	any.

       -d     `Dumb' mode; run all three passes.

       -i isect
	      Only rebuild index section isect;	may  be	 repeated  to  rebuild
	      multiple	sections.  The name is special and just	reads the are-
	      nas.

       -M imemsize
	      The amount of memory, in bytes, to use for caching raw disk  ac-
	      cesses while running buildindex.	(This is not a property	of the
	      created index.)  The usual suffices apply.  The default is 256M.

       Checkindex examines the Venti index described in	venti.conf.  The  pro-
       gram  detects  various  error conditions	including: blocks that are not
       indexed,	index entries for blocks that do not exist, and	duplicate  in-
       dex  entries.   If requested, an	attempt	can be made to fix errors that
       are found.

       The tmp file, usually a disk partition, must be large enough to store a
       copy  of	 the  index.   This temporary space is used to perform a merge
       sort of index entries generated by reading the arenas.

       Options to checkindex are:

       -B blockcachesize
	      The amount of memory, in bytes, to use for caching raw disk  ac-
	      cesses while running checkindex.	The default is 8k.

       -f     Attempt to fix any errors	that are found.

       Checkarenas examines the	Venti arenas contained in the given file.  The
       program detects various error conditions, and  optionally  attempts  to
       fix any errors that are found.

       Options to checkarenas are:

       -a     For each arena, scan the entire data section.  If	this option is
	      omitted, only the	end section of the arena is examined.

       -f     Attempt to fix any errors	that are found.

       -v     Increase the verbosity of	output.

SOURCE
       /src/cmd/venti/srv

SEE ALSO
BUGS
       Buildindex should allow an individual index section to be rebuilt.

								  VENTI-FMT(8)

NAME | SYNOPSIS | DESCRIPTION | SOURCE | SEE ALSO | BUGS

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=venti-fmt&sektion=8&manpath=FreeBSD+12.2-RELEASE+and+Ports>

home | help