Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
mfsmaster.cfg(5)	    This is part of MooseFS	      mfsmaster.cfg(5)

NAME
       mfsmaster.cfg - main configuration file for mfsmaster

DESCRIPTION
       The   file  mfsmaster.cfg  contains  configuration  of  MooseFS	master
       process.

SYNTAX
       Syntax is:

       OPTION =	VALUE

       Lines starting with # character are ignored as comments.

OPTIONS
       Configuration options:

       WORKING_USER
	      user to run daemon as

       WORKING_GROUP
	      group to run daemon as; optional value - if empty	 then  default
	      user group will be used

       SYSLOG_IDENT
	      name  of process to place	in syslog messages; default is mfsmas-
	      ter

       LOCK_MEMORY
	      whether to perform mlockall() to avoid  swapping	out  mfsmaster
	      process; default is 0, i.e. no

       LIMIT_GLIBC_MALLOC_ARENAS
	      limit  malloc arenas to given value - prevents server from using
	      huge amount of virtual memory (Linux onty, default is 4)

       DISABLE_OOM_KILLER
	      disable out of memory killer (Linux only,	default	is 1)

       NICE_LEVEL
	      nice level to run	daemon with; default  is  -19;	note:  process
	      must be started as root to increase priority, if setting of pri-
	      ority fails, process retains the nice level it started with

       FILE_UMASK
	      set default umask	for group and others (user has always 0);  de-
	      fault is 027 - block write for group and block all for others

       DATA_PATH
	      where to store metadata files and	lock file

       EXPORTS_FILENAME
	      alternate	location/name of mfsexports.cfg	file

       TOPOLOGY_FILENAME
	      alternate	location/name of mfstopology.cfg file

       LICENCE_FILENAME
	      alternate	 location/name	of  mfslicence.bin  file  (pro version
	      only)

       BACK_LOGS
	      number of	metadata change	log files (default is 50)

       METADATA_SAVE_FREQ
	      how often	(in hours) master will store metadata (default is 1)

       METADATA_DOWNLOAD_FREQ
	      how often	(in hours) leader will download	metadata from  follow-
	      ers (pro version only ; default is 24)

       METADATA_DOWNLOAD_LIMIT
	      metadata	downloading limit in mebibytes per second (pro version
	      only ; 0 means no	limit, default is 50 - 50MiB/s = ~52.4MB/s)

       BACK_META_KEEP_PREVIOUS
	      number of	previous metadata files	to be kept (default is 1)

       CHANGELOG_PRESERVE_SECONDS
	      how many seconds of change logs have to be preserved  in	memory
	      (default	is 1800; this sets the minimum,	actual number may be a
	      bit bigger due to	logs being kept	in 5k  blocks;	zero  disables
	      extra logs storage)

       CHANGELOG_SAVE_MODE
	      Changelog	save mode (default is 0)
	      0	 -  write  in  background by different process (less safe, but
	      doesn't make master stop in case of heavy	hdd load)
	      1	- write	in foreground without syncing data (master  waits  for
	      every changelog to be saved to hdd, but without syncing -	a lit-
	      tle more safe than the background	option,	but may	 cause	master
	      to stop and wait for flushing hdd	buffers)
	      2	 - write in foreground with fsync after	each write (very safe,
	      but may make your	master very slow unless	you have very  sophis-
	      ticated hardware)

       MISSING_LOG_CAPACITY
	      how  many	 missing  chunks  will	be  stored  in	master	(up to
	      100*MISSING_LOG_CAPACITY bytes of	memory will be allocated ; de-
	      fault value is 100000)

       MATOML_LISTEN_HOST
	      IP  address to listen on for metalogger, masters and supervisors
	      connections (* means any)

       MATOML_LISTEN_PORT
	      port to listen on	for metalogger,	masters	and  supervisors  con-
	      nections

       MASTER_HOST
	      MooseFS master host, IP is allowed only in single-master instal-
	      lations (pro version only	; default is mfsmaster)

       MASTER_RECONNECTION_DELAY
	      delay in seconds before next try to reconnect  to	 master-leader
	      if not connected (pro version only ; default is 5)

       MASTER_TIMEOUT
	      timeout  in  seconds  for	master-leader connections (pro version
	      only ; default is	10)

       BIND_HOST
	      local address to use for connecting with master-leader (pro ver-
	      sion only	; default is *,	i.e. default local address)

       MATOCS_LISTEN_HOST
	      IP  address  to  listen  on for chunkserver connections (* means
	      any)

       MATOCS_LISTEN_PORT
	      port to listen on	for chunkserver	connections

       MATOCS_TIMEOUT
	      default timeout in  seconds  for	master-chunkserver  connection
	      (default is 10)

       AUTH_CODE
	      Optional	 authentication	 string.  When	defined	 -  then  only
	      chunkservers with	the same AUTH_CODE are allowed to  connect  to
	      this  master. When not defined (default) - then all chunkservers
	      are allowed. If you want to switch  on  chunkserver  authentica-
	      tion,  then first	define AUTH_CODE in all	your chunkservers (and
	      reload/restart them) then	define	this  option  and  master  and
	      reload/restart  it.  Remember  that  after reload	currently con-
	      nected chunkservers are NOT disconnected.	New AUTH_CODE will  be
	      used only	when chunkservers will make new	connection.

       REMAP_BITS, REMAP_SOURCE_IP_CLASS, REMAP_DESTINATION_IP_CLASS
	      Optional IP class	remapping. Remap chunkserver IP	addresses with
	      first  REMAP_BITS	 of  IP	  equal	  to   first   REMAP_BITS   of
	      REMAP_SOURCE_IP_CLASS.  During  remapping	 system	will set first
	      REMAP_BITS of IP address to first	REMAP_BITS  of	REMAP_DESTINA-
	      TION_IP_CLASS. (defaults are zeros which means no	remap)

       REPLICATIONS_DELAY_INIT
	      initial  delay  in seconds before	starting replications (default
	      is 60)

       REPLICATIONS_RESPECT_TOPOLOGY
	      whether to make undergoal	replications respect topology (default
	      is 0)
	      0	- do not respect topology
	      1	 -  pick  destination  server  at random, but then choose best
	      source server
	      2	- try to find destination server in the	same rack  as  one  of
	      the  existing  copies  and  then replicate chunk locally (in the
	      same rack)

       CREATIONS_RESPECT_TOPOLOGY
	      whether new chunks should	be recorded with respect  to  topology
	      (default is 0)
	      0	- do not respect topology
	      N	 (N>0)	- first	try to create new chunks on servers with topo-
	      logical distance LOWER than N from the client; if	not  possible,
	      for  example  because of storage class, chunk servers being busy
	      or lacking space,	then try  servers  with	 distance  greater  or
	      equal to N

       CHUNKS_UNIQUE_MODE
	      avoid  using same	ip/rack	for different chunk copies (default is
	      0)
	      0	- ignore ip and	rackid (standard behaviour)
	      1	- avoid	storing	more than one copy on chunkservers using  same
	      IP number
	      2	 -  avoid  storing more	than one copy on chunkservers using IP
	      number from the same rack	id

       CHUNKS_LOOP_MAX_CPS
	      Chunks loop shouldn't check more chunks per seconds  than	 given
	      number (default is 100000)

       CHUNKS_LOOP_MIN_TIME
	      Chunks  loop shouldn't be	done in	less seconds than given	number
	      (default is 300)

       CHUNKS_SOFT_DEL_LIMIT
	      Soft maximum number of chunks to delete on one chunkserver  (de-
	      fault is 10)

       CHUNKS_HARD_DEL_LIMIT
	      Hard  maximum number of chunks to	delete on one chunkserver (de-
	      fault is 25)

       CHUNKS_WRITE_REP_LIMIT
	      Maximum number of	chunks to replicate to	one  chunkserver  (de-
	      fault is 2,1,1,4 - see NOTES)

       CHUNKS_READ_REP_LIMIT
	      Maximum  number of chunks	to replicate from one chunkserver (de-
	      fault is 10,5,2,5	- see NOTES)

       CS_HEAVY_LOAD_THRESHOLD
	      Threshold	for chunkserver	load. (default is 150 -	see NOTES)

       CS_HEAVY_LOAD_RATIO_THRESHOLD
	      Threshold	ratio for chunkserver  load  (default  is  3.0	-  see
	      NOTES)

       CS_HEAVY_LOAD_GRACE_PERIOD
	      Defines  how  long chunkservers will remain in 'grace' mode (de-
	      fault is 900 - see NOTES)

       CS_MAINTENANCE_MODE_TIMEOUT
	      Maximum number of	seconds	server can be in maintenance mode (de-
	      fault value is 0 - which means 'forever')

       CS_TEMP_MAINTENANCE_MODE_TIMEOUT
	      Maximum  number  of seconds server can be	in "temporary" mainte-
	      nance mode (server is switched to	this mode whenever is  stopped
	      gracefully, after	reconnection server is switched	back to	normal
	      mode automatically ; default value: 1800)

       CS_DAYS_TO_REMOVE_UNUSED
	      How many days unused (disconnected) chunkserver should  be  kept
	      in  master  data structures (valid values: 0 - 365 ; 0 means in-
	      definitely ; default value: 7)

       ACCEPTABLE_DIFFERENCE
	      Maximum difference between space usage of	 chunkservers  (depre-
	      cated, use ACCEPTABLE_PERCENTAGE_DIFFERENCE instead)

       ACCEPTABLE_PERCENTAGE_DIFFERENCE
	      Maximum	percentage   difference	  between   space   usage   of
	      chunkservers (default is 1 = 1%)

       PRIORITY_QUEUES_LENGTH
	      Length of	priority queues	(for endangered, undergoal etc.	chunks
	      -	chunks that should be processed	first -	default	is 1000000)

       MATOCL_LISTEN_HOST
	      IP  address to listen on for client (mount) connections (* means
	      any)

       MATOCL_LISTEN_PORT
	      port to listen on	for client (mount) connections

       SESSION_SUSTAIN_TIME
	      How long to sustain a disconnected client	session	 (in  seconds;
	      default is 86400 = 1 day)

       QUOTA_TIME_LIMIT
	      Grace period in secods for soft quota (deprecated, use QUOTA_DE-
	      FAULT_GRACE_PERIOD instead for default value or specify it indi-
	      vidually)

       QUOTA_DEFAULT_GRACE_PERIOD
	      Default  grace  period  in  seconds  for	soft quota (default is
	      604800 = 7 days)

       ATIME_MODE
	      Set atime	modification mode (default is 0	= always modify	 atime
	      -	see NOTES)

       RESERVE_SPACE
	      Set  amount  of  space reserved for superuser (default is	0 = do
	      not reserve space	for superuser -	see NOTES)

       MAX_ALLOWED_HARD_LINKS
	      Define limit for number of hardlinks allowed for one object (de-
	      fault is 32767; possible values are from 8 to 65000)

NOTES
       Chunks  in  master  are tested in a loop. Speed (or frequency) is regu-
       lated by	 two  options  CHUNKS_LOOP_MIN_TIME  and  CHUNKS_LOOP_MAX_CPS.
       First  defines  minimal	time between iterations	of the loop and	second
       defines maximal number of chunk tests per second.  Typically at the be-
       ginning,	when number of chunks is small,	time is	constant, regulated by
       CHUNK_LOOP_MIN_TIME, but	when number of	chunks	beccomes  bigger  then
       time of loop can	increase according to CHUNKS_LOOP_MAX_CPS.

       Example:	CHUNKS_LOOP_MIN_TIME is	set to 300, CHUNKS_LOOP_MAX_CPS	is set
       to 100000 and there is 1000000 (one  million)  chunks  in  the  system.
       1000000/100000 =	10, which is less than 300, so one loop	iteration will
       take 300	seconds.  With 1000000000  (one	 billion)  chunks  the	system
       needs 10000 seconds for one iteration of	the loop.

       Deletion	 limits	are defined as 'soft' and 'hard' limit.	When number of
       chunks to delete	increases from loop to loop, current limit can be tem-
       porary increased	above soft limit, but never above hard limit.

       Replication limits are divided into four	cases:

       o first limit is	for endangered chunks (chunks with only	one copy)

       o second	 limit	is  for	undergoal chunks (chunks with number of	copies
	 lower than specified goal)

       o third limit is	for rebalance between servers with space usage	around
	 arithmetic mean

       o fourth	limit is for rebalance between other servers (very low or very
	 high space usage)

       Usually first number should be grater than or equal to  second,	second
       greater	than  or  equal	 to third, and fourth greater than or equal to
       third ( 1st >= 2nd >= 3rd <= 4th	). If one number is  given,  then  all
       limits are set to this number (for backward compatibility).

       Whenever	 chunkserver  load is higher than CS_HEAVY_LOAD_THRESHOLD  and
       CS_HEAVY_LOAD_RATIO_THRESHOLD times  higher  than  average  load,  then
       chunkserver  is	switched into 'grace' mode. Chunkserver	stays in grace
       mode for	CS_HEAVY_LOAD_GRACE_PERIOD seconds.

       There are five values for ATIME_MODE (all other values are  treated  as
       0):

       o 0 = Always modify atime for files, folders and	symlinks.

       o 1  =  Always  modify  atime  but only in case of files	(do not	modify
	 atime in case of folders and symlinks).

       o 2 = Modify atime only when it is lower	than ctime or mtime  and  when
	 current  time is higher than ctime or mtime respectively, also	modify
	 atime when current atime is older than	24h. Do	 it  for  all  objects
	 during	access (like "relatime"	option in Linux).

       o 3  =  Same as above but only in case of files.	In case	of folders and
	 symlinks do not modify	atime.

       o 4 = Never modify atime	during access (like "noatime" option).

       You can reserve space for superuser using RESERVE_SPACE option. You can
       define  it as number of bytes, percent of total space, capacity of big-
       gest chunkserver, etc.

       o # or #B = number of bytes reserved  for  superuser.  Standard	metric
	 prefixes can be used -	SI and IEC (k,K,M,Mi,G,Gi etc.)

       o #% or #.#% = percent of total capacity	of MooseFS instance

       o #U  or	#.#U = multiplies of "U" value;	U is defined as	maximum	number
	 of bytes currently used by a single chunkserver

       o #C or #.#C = multiplies of "C"	value; C is defined as	maximum	 total
	 capacity of a single chunkserver

       When  your network has two (or more) IP classes you may want to use one
       network for standard communication between  MFS	modules	 and  separate
       network	 only	for  I/O.  It  can  be	done  by  setting  REMAP_BITS,
       REMAP_SOURCE_IP_CLASS and  REMAP_DESTINATION_IP_CLASS.	When  you  set
       these  options  then  master  will  change  internally  IP addresses of
       chunkservers and	will send them as chunk	 locations,  so	 clients  will
       make  connections  with chunkservers using new (destination) IP for all
       I/O, but	still communicate with	master	using  original	 (source)  IP.
       Also  chunkservers will use original IP to communicate with master, but
       they will use new IP's to communicate between themselves	during	repli-
       cation.	Beware	that  all clients and chunkservers must	have access to
       both networks, but masters, metaloggers etc. will need only  access  to
       the source network.

COPYRIGHT
       Copyright (C) 2020 Jakub	Kruszona-Zawadzki, Core	Technology Sp. z o.o.

       This file is part of MooseFS.

       MooseFS	is free	software; you can redistribute it and/or modify	it un-
       der the terms of	the GNU	General	Public License	as  published  by  the
       Free Software Foundation, version 2 (only).

       MooseFS	is distributed in the hope that	it will	be useful, but WITHOUT
       ANY WARRANTY; without even the implied warranty of  MERCHANTABILITY  or
       FITNESS	FOR  A	PARTICULAR PURPOSE. See	the GNU	General	Public License
       for more	details.

       You should have received	a copy of the GNU General Public License along
       with  MooseFS;  if not, write to	the Free Software Foundation, Inc., 51
       Franklin	 St,  Fifth  Floor,  Boston,  MA  02111-1301,  USA  or	 visit
       http://www.gnu.org/licenses/gpl-2.0.html

SEE ALSO
       mfsmaster(8), mfsexports.cfg(5) mfstopology.cfg(5)

MooseFS	3.0.113-1		   May 2020		      mfsmaster.cfg(5)

NAME | DESCRIPTION | SYNTAX | OPTIONS | NOTES | COPYRIGHT | SEE ALSO

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=mfsmaster.cfg&sektion=5&manpath=FreeBSD+12.2-RELEASE+and+Ports>

home | help