Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
VENTI-FMT(8)		    System Manager's Manual		  VENTI-FMT(8)

NAME
       buildindex,  checkarenas, checkindex, conf, fmtarenas, fmtbloom,	fmtin-
       dex, fmtisect, syncindex	- prepare and maintain a venti server

SYNOPSIS
       venti/fmtarenas [ -4Z ] [ -a arenasize ]	[ -b blocksize ] name file

       venti/fmtisect [	-1Z ] [	-b blocksize ] name file

       venti/fmtbloom [	-n nblocks | -N	nhash ]	[ -s size ] file

       venti/fmtindex [	-a ] venti.conf

       venti/conf [ -w ] partition [ configfile	]

       venti/buildindex	[ -bd ]	[ -i isect ] ... [ -M imemsize ] venti.conf

       venti/checkindex	[ -f ] [ -B blockcachesize ] venti.conf	tmp

       venti/checkarenas [ -afv	] file

DESCRIPTION
       These commands aid in the setup,	maintenance, and  debugging  of	 venti
       servers.	 See venti(7) for an overview of the venti system and venti(8)
       for an overview of the data structures used by the venti	server.

       Note that the units for the various sizes in the	following commands can
       be specified by appending or to indicate	kilobytes, megabytes, or giga-
       bytes respectively.

   Formatting
       To prepare a server for its initial use,	the arena partitions  and  the
       index sections must be formatted	individually, with fmtarenas and fmti-
       sect.  Then the collection of index sections must be  combined  into  a
       venti index with	fmtindex.

       Fmtarenas  formats  the given file, typically a disk partition, into an
       arena partition.	 The arenas in the partition are given	names  of  the
       form  name%d, where %d is replaced with a sequential number starting at
       0.

       Options to fmtarenas are:

       -a arenasize
	      The arenas are of	arenasize bytes.  The default is  512M,	 which
	      was  selected  to	provide	a balance between the number of	arenas
	      and the ability to copy an  arena	 to  external  media  such  as
	      recordable CDs and tapes.

       -b blocksize
	      The  size,  in bytes, for	read and write operations to the file.
	      The size is recorded in the file,	and is	used  by  applications
	      that access the arenas.  The default is 8k.

       -4     Create a `version	4' arena partition for backwards compatibility
	      with old servers.	 The default is	version	5, used	by the current
	      venti server.

       -Z     Do  not zero the data sections of	the arenas.  Using this	option
	      reduces the formatting time but should only be used when	it  is
	      known  that  the file was	already	zeroed.	 (Version 4 only; ver-
	      sion 5 sections are not and do not need to be zeroed.)

       Fmtisect	formats	the given file,	typically a disk partition, as a venti
       index section with the specified	name.  Each of the index sections in a
       venti configuration must	have a unique name.

       Options to fmtisect are:

       -b bucketsize
	      The size of an index bucket, in bytes.  All the  index  sections
	      within  a	 index must have the same bucket size.	The default is
	      8k.

       -1     Create a `version	1' index section for  backwards	 compatibility
	      with old servers.	 The default is	version	2, used	by the current
	      venti server.

       -Z     Do not zero the index.  Using this option	reduces	the formatting
	      time  but	should only be used when it is known that the file was
	      already zeroed.  (Version	1 only;	version	2 sections are not and
	      do not need to be	zeroed.)

       Fmtbloom	 formats the given file	as a Bloom filter (see venti(7)).  The
       options are:

       -n nblock | -N nhash
	      The number of blocks expected to be indexed by the filter	or the
	      number  of hash functions	to use.	 If the	-n option is given, it
	      is used, along with the total size of the	filter,	to compute  an
	      appropriate nhash.

       -s size
	      The  size	of the Bloom filter.  The default is the total size of
	      the file.	 In either case, size is rounded down to  a  power  of
	      two.

       The  file  argument in the commands above can be	of the form file:lo-hi
       to specify a range of the file.	Lo and hi are specified	in  bytes  but
       can  have  the usual k, m, or g suffixes.  Either lo or hi may be omit-
       ted.  This notation eliminates the need to partition raw	disks on  non-
       Plan 9 systems.

       Fmtindex	 reads	the  configuration file	venti.conf and initializes the
       index sections to form a	usable index structure.	 The arena  files  and
       index  sections must have previously been formatted using fmtarenas and
       fmtisect	respectively.

       The function of a venti index is	to map a SHA1 fingerprint to  a	 loca-
       tion  in	 the data section of one of the	arenas.	 The index is composed
       of blocks, each of which	contains the mapping for a fixed range of pos-
       sible fingerprint values.  Fmtindex determines the mapping between SHA1
       values and the blocks of	the collection of index	sections.   Once  this
       mapping	has  been  determined, it cannot be changed without rebuilding
       the index.  The basic assumption	in the current implementation is  that
       the index structure is sufficiently empty that individual blocks	of the
       index will rarely overflow.  The	total size  of	the  index  should  be
       about 2%	to 10% of the total size of the	arenas,	but the	exact percent-
       age depends both	on the index block size	and  the  compressed  size  of
       blocks stored.  See the discussion in venti(8) for more.

       Fmtindex	also computes a	mapping	between	a linear address space and the
       data section of the collection of arenas.  The -a option	can be used to
       add  additional	arenas	to an index.  To use this feature, add the new
       arenas to venti.conf after the existing arenas and  then	 run  fmtindex
       -a.

       A  copy	of  the	above mappings is stored in the	header for each	of the
       index sections.	These copies enable buildindex to restore a single in-
       dex section without rebuilding the entire index.

       To  make	 it easier to bootstrap	servers, the configuration file	can be
       stored in otherwise empty space at the beginning	of  any	 venti	parti-
       tions using conf.  A partition so branded with a	configuration file can
       be used in place	of a configuration file	when invoking any of the venti
       commands.   By  default,	conf prints the	configuration stored in	parti-
       tion.  When invoked with	the -w flag, conf reads	a  configuration  file
       from configfile (or else	standard input)	and stores it in partition.

   Checking and	Rebuilding
       Buildindex  populates  the  index  for  the  Venti  system described in
       venti.conf.  The	index must have	previously been	formatted using	fmtin-
       dex.   This  command is typically used to build a new index for a Venti
       system when the old index becomes too small, or to rebuild an index af-
       ter  media failure.  Small errors in an index can usually be fixed with
       checkindex, but checkindex requires a  large  temporary	workspace  and
       buildindex does not.

       Options to buildindex are:

       -b     Reinitialise the Bloom filter, if	any.

       -d     `Dumb' mode; run all three passes.

       -i isect
	      Only  rebuild  index  section  isect; may	be repeated to rebuild
	      multiple sections.  The name is special and just reads the  are-
	      nas.

       -M imemsize
	      The  amount of memory, in	bytes, to use for caching raw disk ac-
	      cesses while running buildindex.	(This is not a property	of the
	      created index.)  The usual suffices apply.  The default is 256M.

       Checkindex  examines the	Venti index described in venti.conf.  The pro-
       gram detects various error conditions including:	blocks	that  are  not
       indexed,	 index entries for blocks that do not exist, and duplicate in-
       dex entries.  If	requested, an attempt can be made to fix  errors  that
       are found.

       The tmp file, usually a disk partition, must be large enough to store a
       copy of the index.  This	temporary space	is used	 to  perform  a	 merge
       sort of index entries generated by reading the arenas.

       Options to checkindex are:

       -B blockcachesize
	      The  amount of memory, in	bytes, to use for caching raw disk ac-
	      cesses while running checkindex.	The default is 8k.

       -f     Attempt to fix any errors	that are found.

       Checkarenas examines the	Venti arenas contained in the given file.  The
       program	detects	 various  error	conditions, and	optionally attempts to
       fix any errors that are found.

       Options to checkarenas are:

       -a     For each arena, scan the entire data section.  If	this option is
	      omitted, only the	end section of the arena is examined.

       -f     Attempt to fix any errors	that are found.

       -v     Increase the verbosity of	output.

SOURCE
       /usr/local/plan9/src/cmd/venti/srv

SEE ALSO
       venti(7), venti(8)

BUGS
       Buildindex should allow an individual index section to be rebuilt.

								  VENTI-FMT(8)

NAME | SYNOPSIS | DESCRIPTION | SOURCE | SEE ALSO | BUGS

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=venti-fmt&sektion=8&manpath=FreeBSD+12.0-RELEASE+and+Ports>

home | help