Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages


home | help
mfsmaster.cfg(5)	    This is part of MooseFS	      mfsmaster.cfg(5)

       mfsmaster.cfg - main configuration file for mfsmaster

       The   file  mfsmaster.cfg  contains  configuration  of  MooseFS	master

       Syntax is:


       Lines starting with # character are ignored as comments.

       Configuration options:

	      user to run daemon as

	      group to run daemon as; optional value - if empty	 then  default
	      user group will be used

	      name  of process to place	in syslog messages; default is mfsmas-

	      whether to perform mlockall() to avoid  swapping	out  mfsmaster
	      process; default is 0, i.e. no

	      limit  malloc arenas to given value - prevents server from using
	      huge amount of virtual memory (Linux onty, default is 4)

	      disable out of memory killer (Linux only,	default	is 1)

	      nice level to run	daemon with; default  is  -19;	note:  process
	      must be started as root to increase priority, if setting of pri-
	      ority fails, process retains the nice level it started with

	      set default umask	for group and others (user has always 0);  de-
	      fault is 027 - block write for group and block all for others

	      where to store metadata files and	lock file

	      alternate	location/name of mfsexports.cfg	file

	      alternate	location/name of mfstopology.cfg file

	      alternate	 location/name	of  mfslicence.bin  file  (pro version

	      number of	metadata change	log files (default is 50)

	      how often	(in hours) master will store metadata (default is 1)

	      how often	(in hours) leader will download	metadata from  follow-
	      ers (pro version only ; default is 24)

	      metadata	downloading limit in mebibytes per second (pro version
	      only ; 0 means no	limit, default is 50 - 50MiB/s = ~52.4MB/s)

	      number of	previous metadata files	to be kept (default is 1)

	      how many seconds of change logs have to be preserved  in	memory
	      (default	is 1800; this sets the minimum,	actual number may be a
	      bit bigger due to	logs being kept	in 5k  blocks;	zero  disables
	      extra logs storage)

	      Changelog	save mode (default is 0)
	      0	 -  write  in  background by different process (less safe, but
	      doesn't make master stop in case of heavy	hdd load)
	      1	- write	in foreground without syncing data (master  waits  for
	      every changelog to be saved to hdd, but without syncing -	a lit-
	      tle more safe than the background	option,	but may	 cause	master
	      to stop and wait for flushing hdd	buffers)
	      2	 - write in foreground with fsync after	each write (very safe,
	      but may make your	master very slow unless	you have very  sophis-
	      ticated hardware)

	      how  many	 missing  chunks  will	be  stored  in	master	(up to
	      100*MISSING_LOG_CAPACITY bytes of	memory will be allocated ; de-
	      fault value is 100000)

	      IP  address to listen on for metalogger, masters and supervisors
	      connections (* means any)

	      port to listen on	for metalogger,	masters	and  supervisors  con-

	      MooseFS master host, IP is allowed only in single-master instal-
	      lations (pro version only	; default is mfsmaster)

	      delay in seconds before next try to reconnect  to	 master-leader
	      if not connected (pro version only ; default is 5)

	      timeout  in  seconds  for	master-leader connections (pro version
	      only ; default is	10)

	      local address to use for connecting with master-leader (pro ver-
	      sion only	; default is *,	i.e. default local address)

	      IP  address  to  listen  on for chunkserver connections (* means

	      port to listen on	for chunkserver	connections

	      default timeout in  seconds  for	master-chunkserver  connection
	      (default is 10)

	      Optional	 authentication	 string.  When	defined	 -  then  only
	      chunkservers with	the same AUTH_CODE are allowed to  connect  to
	      this  master. When not defined (default) - then all chunkservers
	      are allowed. If you want to switch  on  chunkserver  authentica-
	      tion,  then first	define AUTH_CODE in all	your chunkservers (and
	      reload/restart them) then	define	this  option  and  master  and
	      reload/restart  it.  Remember  that  after reload	currently con-
	      nected chunkservers are NOT disconnected.	New AUTH_CODE will  be
	      used only	when chunkservers will make new	connection.

	      Optional IP class	remapping. Remap chunkserver IP	addresses with
	      first  REMAP_BITS	 of  IP	  equal	  to   first   REMAP_BITS   of
	      REMAP_SOURCE_IP_CLASS.  During  remapping	 system	will set first
	      REMAP_BITS of IP address to first	REMAP_BITS  of	REMAP_DESTINA-
	      TION_IP_CLASS. (defaults are zeros which means no	remap)

	      initial  delay  in seconds before	starting replications (default
	      is 60)

	      whether to make undergoal	replications respect topology (default
	      is 0)
	      0	- do not respect topology
	      1	 -  pick  destination  server  at random, but then choose best
	      source server
	      2	- try to find destination server in the	same rack  as  one  of
	      the  existing  copies  and  then replicate chunk locally (in the
	      same rack)

	      whether new chunks should	be recorded with respect  to  topology
	      (default is 0)
	      0	- do not respect topology
	      N	 (N>0)	- first	try to create new chunks on servers with topo-
	      logical distance LOWER than N from the client; if	not  possible,
	      for  example  because of storage class, chunk servers being busy
	      or lacking space,	then try  servers  with	 distance  greater  or
	      equal to N

	      avoid  using same	ip/rack	for different chunk copies (default is
	      0	- ignore ip and	rackid (standard behaviour)
	      1	- avoid	storing	more than one copy on chunkservers using  same
	      IP number
	      2	 -  avoid  storing more	than one copy on chunkservers using IP
	      number from the same rack	id

	      Chunks loop shouldn't check more chunks per seconds  than	 given
	      number (default is 100000)

	      Chunks  loop shouldn't be	done in	less seconds than given	number
	      (default is 300)

	      Soft maximum number of chunks to delete on one chunkserver  (de-
	      fault is 10)

	      Hard  maximum number of chunks to	delete on one chunkserver (de-
	      fault is 25)

	      Maximum number of	chunks to replicate to	one  chunkserver  (de-
	      fault is 2,1,1,4 - see NOTES)

	      Maximum  number of chunks	to replicate from one chunkserver (de-
	      fault is 10,5,2,5	- see NOTES)

	      Threshold	for chunkserver	load. (default is 150 -	see NOTES)

	      Threshold	ratio for chunkserver  load  (default  is  3.0	-  see

	      Defines  how  long chunkservers will remain in 'grace' mode (de-
	      fault is 900 - see NOTES)

	      Maximum number of	seconds	server can be in maintenance mode (de-
	      fault value is 0 - which means 'forever')

	      Maximum  number  of seconds server can be	in "temporary" mainte-
	      nance mode (server is switched to	this mode whenever is  stopped
	      gracefully, after	reconnection server is switched	back to	normal
	      mode automatically ; default value: 1800)

	      How many days unused (disconnected) chunkserver should  be  kept
	      in  master  data structures (valid values: 0 - 365 ; 0 means in-
	      definitely ; default value: 7)

	      Maximum difference between space usage of	 chunkservers  (depre-

	      Maximum	percentage   difference	  between   space   usage   of
	      chunkservers (default is 1 = 1%)

	      Length of	priority queues	(for endangered, undergoal etc.	chunks
	      -	chunks that should be processed	first -	default	is 1000000)

	      IP  address to listen on for client (mount) connections (* means

	      port to listen on	for client (mount) connections

	      How long to sustain a disconnected client	session	 (in  seconds;
	      default is 86400 = 1 day)

	      Grace period in secods for soft quota (deprecated, use QUOTA_DE-
	      FAULT_GRACE_PERIOD instead for default value or specify it indi-

	      Default  grace  period  in  seconds  for	soft quota (default is
	      604800 = 7 days)

	      Set atime	modification mode (default is 0	= always modify	 atime
	      -	see NOTES)

	      Set  amount  of  space reserved for superuser (default is	0 = do
	      not reserve space	for superuser -	see NOTES)

	      Define limit for number of hardlinks allowed for one object (de-
	      fault is 32767; possible values are from 8 to 65000)

	      Delay time in seconds after which	inodes of deleted objects will
	      be reused. BE AWARE if you change	this value below 86400 (1 day)
	      you  MUST	 ensure	that this value	is higher than any of the fol-
	      lowing timeouts in all clients: mfsattrcacheto, mfsxattrcacheto,
	      mfsentrycacheto, mfsdirentrycacheto, mfsnegentrycacheto, mfssym-
	      linkcacheto. (default is 86400; possible values are from 300  to

       Chunks  in  master  are tested in a loop. Speed (or frequency) is regu-
       lated by	 two  options  CHUNKS_LOOP_MIN_TIME  and  CHUNKS_LOOP_MAX_CPS.
       First  defines  minimal	time between iterations	of the loop and	second
       defines maximal number of chunk tests per second.  Typically at the be-
       ginning,	when number of chunks is small,	time is	constant, regulated by
       CHUNK_LOOP_MIN_TIME, but	when number of	chunks	beccomes  bigger  then
       time of loop can	increase according to CHUNKS_LOOP_MAX_CPS.

       Example:	CHUNKS_LOOP_MIN_TIME is	set to 300, CHUNKS_LOOP_MAX_CPS	is set
       to 100000 and there is 1000000 (one  million)  chunks  in  the  system.
       1000000/100000 =	10, which is less than 300, so one loop	iteration will
       take 300	seconds.  With 1000000000  (one	 billion)  chunks  the	system
       needs 10000 seconds for one iteration of	the loop.

       Deletion	 limits	are defined as 'soft' and 'hard' limit.	When number of
       chunks to delete	increases from loop to loop, current limit can be tem-
       porary increased	above soft limit, but never above hard limit.

       Replication limits are divided into four	cases:

       o first limit is	for endangered chunks (chunks with only	one copy)

       o second	 limit	is  for	undergoal chunks (chunks with number of	copies
	 lower than specified goal)

       o third limit is	for rebalance between servers with space usage	around
	 arithmetic mean

       o fourth	limit is for rebalance between other servers (very low or very
	 high space usage)

       Usually first number should be grater than or equal to  second,	second
       greater	than  or  equal	 to third, and fourth greater than or equal to
       third ( 1st >= 2nd >= 3rd <= 4th	). If one number is  given,  then  all
       limits are set to this number (for backward compatibility).

       Whenever	 chunkserver  load is higher than CS_HEAVY_LOAD_THRESHOLD  and
       CS_HEAVY_LOAD_RATIO_THRESHOLD times  higher  than  average  load,  then
       chunkserver  is	switched into 'grace' mode. Chunkserver	stays in grace
       mode for	CS_HEAVY_LOAD_GRACE_PERIOD seconds.

       There are five values for ATIME_MODE (all other values are  treated  as

       o 0 = Always modify atime for files, folders and	symlinks.

       o 1  =  Always  modify  atime  but only in case of files	(do not	modify
	 atime in case of folders and symlinks).

       o 2 = Modify atime only when it is lower	than ctime or mtime  and  when
	 current  time is higher than ctime or mtime respectively, also	modify
	 atime when current atime is older than	24h. Do	 it  for  all  objects
	 during	access (like "relatime"	option in Linux).

       o 3  =  Same as above but only in case of files.	In case	of folders and
	 symlinks do not modify	atime.

       o 4 = Never modify atime	during access (like "noatime" option).

       You can reserve space for superuser using RESERVE_SPACE option. You can
       define  it as number of bytes, percent of total space, capacity of big-
       gest chunkserver, etc.

       o # or #B = number of bytes reserved  for  superuser.  Standard	metric
	 prefixes can be used -	SI and IEC (k,K,M,Mi,G,Gi etc.)

       o #% or #.#% = percent of total capacity	of MooseFS instance

       o #U  or	#.#U = multiplies of "U" value;	U is defined as	maximum	number
	 of bytes currently used by a single chunkserver

       o #C or #.#C = multiplies of "C"	value; C is defined as	maximum	 total
	 capacity of a single chunkserver

       When  your network has two (or more) IP classes you may want to use one
       network for standard communication between  MFS	modules	 and  separate
       network	 only	for  I/O.  It  can  be	done  by  setting  REMAP_BITS,
       these  options  then  master  will  change  internally  IP addresses of
       chunkservers and	will send them as chunk	 locations,  so	 clients  will
       make  connections  with chunkservers using new (destination) IP for all
       I/O, but	still communicate with	master	using  original	 (source)  IP.
       Also  chunkservers will use original IP to communicate with master, but
       they will use new IP's to communicate between themselves	during	repli-
       cation.	Beware	that  all clients and chunkservers must	have access to
       both networks, but masters, metaloggers etc. will need only  access  to
       the source network.

       Copyright (C) 2021 Jakub	Kruszona-Zawadzki, Core	Technology Sp. z o.o.

       This file is part of MooseFS.

       MooseFS	is free	software; you can redistribute it and/or modify	it un-
       der the terms of	the GNU	General	Public License	as  published  by  the
       Free Software Foundation, version 2 (only).

       MooseFS	is distributed in the hope that	it will	be useful, but WITHOUT
       ANY WARRANTY; without even the implied warranty of  MERCHANTABILITY  or
       FITNESS	FOR  A	PARTICULAR PURPOSE. See	the GNU	General	Public License
       for more	details.

       You should have received	a copy of the GNU General Public License along
       with  MooseFS;  if not, write to	the Free Software Foundation, Inc., 51
       Franklin	 St,  Fifth  Floor,  Boston,  MA  02111-1301,  USA  or	 visit

       mfsmaster(8), mfsexports.cfg(5) mfstopology.cfg(5)

MooseFS	3.0.116-1		  August 2021		      mfsmaster.cfg(5)


Want to link to this manual page? Use this URL:

home | help