Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
VENTI(8)		    System Manager's Manual		      VENTI(8)

NAME
       venti - archival	storage	server

SYNOPSIS
       venti/venti  [ -Ldrs ] [	-a address ] [ -B blockcachesize ] [ -c	config
	 ] [ -C	lumpcachesize ]	[ -h httpaddress ] [ -I	indexcachesize ] [  -W
	 webroot ]

DESCRIPTION
       Venti  is a SHA1-addressed archival storage server.  See	venti(7) for a
       full introduction to the	system.	 This page documents the structure and
       operation of the	server.

       A  venti	 server	 requires  multiple  disks or disk partitions, each of
       which must be properly formatted	before the server can be run.

   Disk
       The venti server	maintains three	disk structures, typically  stored  on
       raw  disk partitions: the append-only data log, which holds, in sequen-
       tial order, the contents	of every block written to the server; the  in-
       dex,  which  helps  locate a block in the data log given	its score; and
       optionally the bloom filter, a concise  summary	of  which  scores  are
       present in the index.  The data log is the primary storage.  To improve
       the robustness, it should be stored on  a  device  that	provides  RAID
       functionality.	The  index  and	the bloom filter are optimizations em-
       ployed to access	the data log efficiently and can be rebuilt if lost or
       damaged.

       The  data log is	logically split	into sections called arenas, typically
       sized for easy offline backup (e.g., 500MB).  A data log	 may  comprise
       many  disks,  each  storing  one	or more	arenas.	 Such disks are	called
       arena partitions.  Arena	partitions are filled in the  order  given  in
       the configuration.

       The  index  is  logically split into block-sized	pieces called buckets,
       each of which is	responsible for	a particular range of scores.  An  in-
       dex  may	 be  split across many disks, each storing many	buckets.  Such
       disks are called	index sections.

       The index must be sized so that no  bucket  is  full.   When  a	bucket
       fills,  the  server must	be shut	down and the index made	larger.	 Since
       scores appear random, each bucket will contain approximately  the  same
       number  of  entries.  Index entries are 40 bytes	long.  Assuming	that a
       typical block being written to the server is 8192 bytes and  compresses
       to  4096	 bytes,	the active index is expected to	be about 1% of the ac-
       tive data log.  Storing smaller blocks  increases  the  relative	 index
       footprint;  storing  larger blocks decreases it.	 To allow variation in
       both block size and the random distribution of scores to	 buckets,  the
       suggested index size is 5% of the active	data log.

       The  (optional)	bloom  filter is a large bitmap	that is	stored on disk
       but also	kept completely	in memory while	the  venti  server  runs.   It
       helps  the  venti server	efficiently detect scores that are not already
       stored in the index.  The bloom filter starts out zeroed.   Each	 score
       recorded	 in  the bloom filter is hashed	to choose nhash	bits to	set in
       the bloom filter.  A score is definitely	not stored in the index	of any
       of  its	nhash bits are not set.	 The bloom filter thus has two parame-
       ters: nhash (maximum 32)	and the	total bitmap size (maximum 512MB,  232
       bits).

       The  bloom  filter  should  be sized so that nhash x nblock <= 0.7 x b,
       where nblock is the expected number of blocks stored on the server  and
       b  is  the  bitmap  size	in bits.  The false positive rate of the bloom
       filter when sized this way is approximately 2-nblock.  Nhash less  than
       10  are	not very useful; nhash greater than 24 are probably a waste of
       memory.	Fmtbloom (see venti-fmt(8))  can  be  given  either  nhash  or
       nblock; if given	nblock,	it will	derive an appropriate nhash.

   Memory
       Venti  can  make	 effective  use	of large amounts of memory for various
       caches.

       The lump	cache holds recently-accessed venti  data  blocks,  which  the
       server  refers  to as lumps.  The lump cache should be at least 1MB but
       can profitably be much larger.  The lump	cache can be thought of	as the
       level-1	cache:	read  requests handled by the lump cache can be	served
       instantly.

       The block cache holds recently-accessed disk blocks from	the arena par-
       titions.	  The  block cache needs to be able to simultaneously hold two
       blocks from each	arena  plus  four  blocks  for	the  currently-filling
       arena.	The  block  cache can be thought of as the level-2 cache: read
       requests	handled	by the block cache are slower than  those  handled  by
       the lump	cache, since the lump data must	be extracted from the raw disk
       blocks and possibly decompressed, but no	disk accesses are necessary.

       The index cache holds recently-accessed or  prefetched  index  entries.
       The  index  cache  needs	 to be able to hold index entries for three or
       four arenas, at least, in order for prefetching to work properly.  Each
       index  entry is 50 bytes.  Assuming 500MB arenas	of 128,000 blocks that
       are 4096	bytes each after compression, the minimum index	cache size  is
       about  6MB.   The  index	 cache can be thought of as the	level-3	cache:
       read requests handled by	the index cache	must still go to disk to fetch
       the arena blocks, but the costly	random access to the index is avoided.

       The  size  of the index cache determines	how long venti can sustain its
       `burst' write throughput, during	which time the only disk  accesses  on
       the  critical  path are sequential writes to the	arena partitions.  For
       example,	if you want to be able to sustain 10MB/s for an	hour, you need
       enough  index  cache  to	 hold  entries	for  36GB of blocks.  Assuming
       8192-byte blocks, you need room for almost five million index  entries.
       Since  index  entries are 50 bytes each,	you need 250MB of index	cache.
       If the background index update process can make a single	 pass  through
       the  index  in  an  hour,  which	 is possible, then you can sustain the
       10MB/s indefinitely (at least until the arenas are all filled).

       The bloom filter	requires memory	equal to its size  on  disk,  as  dis-
       cussed above.

       A  reasonable  starting	allocation  is	to  divide  memory equally (in
       thirds) between the bloom filter, the index cache,  and	the  lump  and
       block  caches;  the  third  of  memory  allocated to the	lump and block
       caches should be	split unevenly,	with more (say,	two thirds)  going  to
       the block cache.

   Network
       The  venti  server  announces two network services, one (conventionally
       TCP port	venti, 17034) serving  the  venti  protocol  as	 described  in
       venti(7), and one serving HTTP (conventionally TCP port http, 80).

       The  venti  web server provides the following URLs for accessing	status
       information:

       /index A	summary	of the usage of	the arenas and index sections.

       /xindex
	      An XML version of	/index.

       /storage
	      Brief storage totals.

       /set   Disable the values of all	variables.  Variables  are:  compress,
	      whether  or  not	to  compress  blocks (for debugging); logging,
	      whether to write entries to the debugging	logs;  stats,  whether
	      to  collect  run-time  statistics;  icachesleeptime, the time in
	      milliseconds between successive updates of megabytes of the  in-
	      dex  cache;  arenasumsleeptime, the time in milliseconds between
	      reads while checksumming an arena	in the	background.   The  two
	      sleep times should be (but are not) managed by venti; they exist
	      to provide more experience with their effects.  The other	 vari-
	      ables exist only for debugging and performance measurement.

       /set?name=variable
	      Show the current setting of variable.

       /set?name=variable&value=value
	      Set variable to value.

       /graph/name/param/param/...
	      A	 PNG  image  graphing  the named run-time statistic over time.
	      The details  of  names  and  parameters  are  undocumented;  see
	      httpd.c in the venti sources.

       /log   A	list of	all debugging logs present in the server's memory.

       /log/name
	      The contents of the debugging log	with the given name.

       /flushicache
	      Force  venti to begin flushing the index cache to	disk.  The re-
	      quest response will not be sent until the	flush has completed.

       /flushdcache
	      Force venti to begin flushing the	arena  block  cache  to	 disk.
	      The  request  response will not be sent until the	flush has com-
	      pleted.

       Requests	for other files	are served by consulting a directory named  in
       the configuration file (see webroot below).

   Configuration File
       A  venti	 configuration	file enumerates	the various index sections and
       arenas that constitute a	venti system.  The components are indicated by
       the name	of the file, typically a disk partition, in which they reside.
       The configuration file is the only location that	file names  are	 used.
       Internally, venti uses the names	assigned when the components were for-
       matted with fmtarenas or	fmtisect (see venti-fmt(8)).   In  particular,
       only the	configuration needs to be changed if a component is moved to a
       different file.

       The configuration file consists of lines	in the form  described	below.
       Lines starting with # are comments.

       index name
	      Names the	index for the system.

       arenas file
	      File is an arena partition, formatted using fmtarenas.

       isect file
	      File is an index section,	formatted using	fmtisect.

       bloom file
	      File is a	bloom filter, formatted	using fmtbloom.

       After formatting	a venti	system using fmtindex, the order of arenas and
       index sections should not be changed.  Additional  arenas  can  be  ap-
       pended  to  the	configuration; run fmtindex with the -a	flag to	update
       the index.

       The configuration file also  holds  configuration  parameters  for  the
       venti server itself.  These are:

       mem size
	      lump cache size

       bcmem size
	      block cache size

       icmem size
	      index cache size

       addr netaddr
	      network address to announce venti	service	(default tcp!*!venti)

       httpaddr	netaddr
	      network address to announce HTTP service (default	tcp!*!http)

       queuewrites
	      queue writes in memory (default is not to	queue)

       webroot dir
	      directory	tree containing	files for venti's internal HTTP	server
	      to consult for unrecognized URLs

       The units for the various cache sizes above can be specified by append-
       ing  a or (case-insensitive) to indicate	kilobytes, megabytes, or giga-
       bytes respectively.

       The file	name in	the configuration lines	 above	can  be	 of  the  form
       file:lo-hi  to specify a	range of the file.  Lo and hi are specified in
       bytes but can have the usual k, m, or g suffixes.  Either lo or hi  may
       be  omitted.   This notation eliminates the need	to partition raw disks
       on non-Plan 9 systems.

   Command Line
       Many of the options to Venti duplicate parameters that can be specified
       in  the	configuration  file.   The command line	options	override those
       found in	a configuration	file.  Additional options are:

       -c config
	      The server configuration file (default venti.conf)

       -d     Produce various debugging	information on	standard  error.   Im-
	      plies -s.

       -L     Enable  logging.	 By  default all logging is disabled.  Logging
	      slows server operation considerably.

       -r     Allow only read access to	the venti data.

       -s     Do not run in the	background.  Normally, the foreground  process
	      will  exit  once	the  Venti server is initialized and ready for
	      connections.

EXAMPLE
       A simple	configuration:

	      %	cat venti.conf
	      index main
	      isect /tmp/disks/isect0
	      isect /tmp/disks/isect1
	      arenas /tmp/disks/arenas
	      bloom /tmp/disks/bloom
	      mem 10M
	      bcmem 20M
	      icmem 30M
	      %

       Format the index	sections, the arena partition, the bloom  filter,  and
       finally the main	index:

	      %	venti/fmtisect isect0. /tmp/disks/isect0
	      %	venti/fmtisect isect1. /tmp/disks/isect1
	      %	venti/fmtarenas	arenas0. /tmp/disks/arenas &
	      %	venti/fmtbloom /tmp/disks/bloom	&
	      %	wait
	      %	venti/fmtindex venti.conf
	      %

       Start the server	and check the storage statistics:

	      %	venti/venti
	      %	hget http://$sysname/storage

SOURCE
       /usr/local/plan9/src/cmd/venti/srv

SEE ALSO
       venti(1), venti(3), venti(7), venti-backup(8) venti-fmt(8)
       Sean  Quinlan  and  Sean	 Dorward,  ``Venti: a new approach to archival
       storage'', Usenix Conference on File and	Storage	Technologies , 2002.

BUGS
       Setting up a venti server is too	complicated.

       Venti should not	require	the user to decide how to partition its	memory
       usage.

       Users  of  shells  other	than rc(1) will	not be able to use the program
       names shown.  One solution is to	 define	 V=$PLAN9/bin/venti  and  then
       substitute $V/ for venti/ in the	paths above.

								      VENTI(8)

NAME | SYNOPSIS | DESCRIPTION | EXAMPLE | SOURCE | SEE ALSO | BUGS

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=venti&sektion=8&manpath=FreeBSD+12.0-RELEASE+and+Ports>

home | help