Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
knl.conf(5)		   Slurm Configuration File		   knl.conf(5)

NAME
       knl.conf	 -  Slurm configuration	file for Intel Knights Landing proces-
       sor.

DESCRIPTION
       This ASCII file which describes	configuration  information  for	 Intel
       Knights	Landing	 processors  and its name may depend upon the NodeFea-
       tures plugin configured in Slurm. For example, on Cray systems NodeFea-
       tures  should  be  configured  to "knl_cray" and	its configuration file
       will be read from "knl_cray.conf".  The file location can  be  modified
       at  system build	time using the DEFAULT_SLURM_CONF parameter or at exe-
       cution time by setting the SLURM_CONF environment  variable.  The  file
       will  always  be	 located in the	same directory as the slurm.conf file.
       This file is optional.

       Parameter names are case	insensitive.  Any text following a "#" in  the
       configuration  file  is	treated	 as  a comment through the end of that
       line.  Changes to the configuration file	take effect  upon  restart  of
       Slurm daemons, daemon receipt of	the SIGHUP signal, or execution	of the
       command "scontrol reconfigure" unless otherwise noted.

       The overall configuration parameters available include:

       AllowMCDRAM
	      Specify the MCDRAM modes which jobs are allowed  to  use.	  This
	      may  be  a subset	of MCDRAM modes	supported by the node.	If not
	      specified, all MCDRAM modes supported by the node	are  available
	      for  use.	  The comma separated list of allowed MCDRAM modes may
	      include any of the modes listed below.

	      cache	       All of MCDRAM to	be used	as cache.

	      equal	       MCDRAM to be used partly	as  cache  and	partly
			       combined	with primary memory.

	      flat	       MCDRAM  to be combined with primary memory into
			       a "flat"	memory space.

       AllowNUMA
	      Specify the NUMA modes which jobs	are allowed to use.  This  may
	      be  a subset of NUMA modes supported by the node.	 If not	speci-
	      fied, all	NUMA modes supported by	the  node  are	available  for
	      use.  The	comma separated	list of	allowed	NUMA modes may include
	      any of the modes listed below.  Note that	Slurm can only support
	      homogeneous nodes	(e.g. the same number of cores per NUMA	node).
	      KNL scn4 and quad	modes are not homogeneous, but each each  NUMA
	      mode will	have either 16 or 18 cores.  This will result in Slurm
	      using the	lower core count and finding a total  of  256  threads
	      rather  than  272	 threads and setting the node to a DOWN	state.
	      Therefore	it is recommended that snc4 and	quad mode not  be  al-
	      lowed at this time.

	      a2a	       All to all

	      snc2	       Sub-NUMA	cluster	2

	      snc4	       Sub-NUMA	cluster	4

	      hemi	       Hemisphere

	      quad	       Quadrant

       AllowUserBoot
	      A	 comma	delimited list of users	allowed	to modify a node's MC-
	      DRAM or NUMA state.  If not specified then any user can change a
	      node's state and reboot it.

       BootTime
	      Estimated	time to	reboot a node in seconds.  Used	as a basis for
	      optimizing scheduling decisions.	The default value is 300  sec-
	      onds  (5	minutes) for the "knl_generic" plugin and 2700 seconds
	      (45 minutes) for the "knl_cray" plugin.

       CapmcPath
	      Fully qualified path to the capmc	program.  The default value is
	      "/opt/cray/capmc/default/bin/capmc".   This  parameter  is  used
	      only by the "knl_cray" plugin.

       CapmcPollFreq
	      Time interval between when the capmc  program  should  poll  for
	      node  state  changes,  in	seconds.  The default value is 45 sec-
	      onds.  This parameter is used only by the	"knl_cray" plugin.

       CapmcRetries
	      Number of	times to retry failed operations of the	capmc program.
	      Default value is 4.

       CapmcTimeout
	      Time  limit  for	the capmc program to return status information
	      milliseconds.  The default value is 60000	milliseconds  and  the
	      minimum  value  is 1000 milliseconds.  This parameter is used by
	      the "knl_cray" plugin, plus the capmc_suspend  and  capmc_resume
	      programs used for	suspending and resuming	nodes.

       CnselectPath
	      Fully qualified path to the cnselect program.  The default value
	      is "/opt/cray/sdb/default/bin/cnselect".	This parameter is used
	      only by the "knl_cray" plugin.

       DefaultMCDRAM
	      Specify  the default MCDRAM modes	for job's which	do not specify
	      a	value.	This is	only used when a node is booted	 and  the  job
	      which has	been allocated the node	does not specify a desired MC-
	      DRAM mode.  The value can	include	one  of	 the  possible	values
	      identified  with	the AllowMCDRAM	configuration parameter	above.
	      The default value	is "cache".

       DefaultNUMA
	      Specify the default NUMA modes for job's which do	not specify  a
	      value.   This  is	 only  used  when a node is booted and the job
	      which has	been allocated the node	does  not  specify  a  desired
	      NUMA  mode.   The	 value	can include one	of the possible	values
	      identified with the  AllowNUMA  configuration  parameter	above.
	      The default value	is "a2a".

       Force  If  set  to a non-zero value then	load the node_features/generic
	      plugin even on non-KNL nodes.  Used primarily for	 testing  pur-
	      poses.

       LogFile
	      Fully qualified path to a	log file.  The default value is	Slurm-
	      ctldLogFile from the slurm.conf configuration file.  This	is op-
	      tion is used only	by the campc_suspend and campc_resume programs
	      (which power down	and reboot nodes in the	appropriate configura-
	      tion).

       McPath Fully qualified path to memory controller	device file directory.
	      Children	of   this   directory	with   names   of   the	  form
	      "mc#/csrow#/ue_count"  (i.e.  the	 count of unrecoverable	memory
	      errors) will be monitored	for non-zero values.  If  such	errors
	      are  detected,  the  node	 will  be  set to a DOWN state and the
	      slurmd daemon will shutdown.  The	 default  value	 is  "/sys/de-
	      vices/system/edac/mc".  See also UmeCheckInterval.

       NodeRebootWeight
	      If  a  compute node requires a reboot to be usable for a pending
	      job, then	reset the node's weight	to the specified  value.   The
	      default  value is	4,294,967,294 (0xfffffffe).  See also "Weight"
	      in the node configuration	specification of slurm.conf.

       NumaCpuBind
	      Contains pairs of	NUMA modes and the CpuBind mode	to set a  node
	      to  for  that  mode.   Any compute node found with or set	to the
	      specified	NUMA mode will have that node's	CpuBind	field  set  to
	      the  configured  value.	The  NUMA  node	will be	followed by an
	      equal sign the desired CpuBind mode for that NUMA	mode. Multiple
	      NUMA  mode  and CpuBind modes should be in a semicolon separated
	      list.  By	default	changes	to a node's NUMA mode will not	effect
	      that node's CpuBind mode.	 See the example below.

       SyscfgPath
	      Fully qualified path to Intel's syscfg program, which identifies
	      current KNL configuration	by viewing BIOS	settings.  If not  de-
	      fined,  the current BIOS setting will not	be available.  The de-
	      fault value is "/usr/bin/syscfg".	 This parameter	is  used  only
	      by the "knl_generic" plugin.

       SyscfgTimeout
	      Timeout  for  syscfg  program in milliseconds.  Default value is
	      1000 milliseconds.  For Dell KNL systems,	experience  has	 shown
	      that a higher value of 10000 milliseconds	is more	appropriate.

       SystemType
	      Used to distinguish the flavor of	knl we are dealing with.  Pos-
	      sible options are	"Dell" and "Intel".  The default value is "In-
	      tel".  This parameter is used only by the	"knl_generic" plugin.

       UmeCheckInterval
	      Interval,	in microseconds, between checks	for Uncorrectable Mem-
	      ory Errors (UME).	 If such errors	are detected, the node will be
	      set  to  a  DOWN state and the slurmd daemon will	shutdown.  The
	      default value is 0 (disabled).  See also McPath.

       ValidateMode
	      If set to	1 then validate, but do	not modify the node's  config-
	      ured  MCDRAM and NUMA modes from the slurm.conf file. If the ac-
	      tual modes do not	match configured values	the node will  be  set
	      to  a  DOWN  state.  Every KNL nodes MCDRAM and NUMA states must
	      both be listed in	the slurm.conf file.  This parameter  is  used
	      only by the "knl_cray" plugin.

EXAMPLE
       ###################################################################
       # knl_cray.conf
       # Slurm configuration file for Intel Knights Landing on Cray system
       ###################################################################
       CapmcPath=/opt/cray/capmc/default/bin/capmc
       CapmcTimeout=6000
       DefaultMCDRAM=flat
       DefaultNUMA=a2a
       NumaCpuBind=a2a=core;snc2=thread;snc4=thread
       LogFile=/var/tmp/slurm_node_feature.log
       SyscfgPath=/usr/sbin/syscfg

COPYING
       Copyright (C) 2015-2017 SchedMD LLC.

       This  file  is  part  of	Slurm, a resource management program.  For de-
       tails, see <https://slurm.schedmd.com/>.

       Slurm is	free software; you can redistribute it and/or modify it	 under
       the  terms  of  the GNU General Public License as published by the Free
       Software	Foundation; either version 2 of	the License, or	(at  your  op-
       tion) any later version.

       Slurm  is  distributed  in the hope that	it will	be useful, but WITHOUT
       ANY WARRANTY; without even the implied warranty of  MERCHANTABILITY  or
       FITNESS	FOR  A PARTICULAR PURPOSE.  See	the GNU	General	Public License
       for more	details.

SEE ALSO
       slurm.conf(5)

May 2018		   Slurm Configuration File		   knl.conf(5)

NAME | DESCRIPTION | EXAMPLE | COPYING | SEE ALSO

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=knl.conf&sektion=5&manpath=FreeBSD+12.2-RELEASE+and+Ports>

home | help