Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages


home | help
ORTE_HOSTS(7)			   Open	MPI			 ORTE_HOSTS(7)

       ORTE_HOSTS  - OpenRTE Hostfile and HOST Behavior: Overview of OpenRTE's
       support for user-supplied hostfiles and comma-delimited lists of	hosts

       OpenRTE supports	several	levels of user-specified host lists  based  on
       an  established	precedence order. Users	can specify a default hostfile
       that contains a list of nodes available to all  app_contexts  given  on
       the  command  line.  Only  one default hostfile can be provided for any
       job. In addition, users can specify a hostfile that contains a list  of
       nodes to	be used	for a specific app_context, or can provide a comma-de-
       limited list of nodes to	be used	for that  app_context  via  the	 -host
       command line option.

       The  precedence	order applied to these various options depends to some
       extent on the local environment.	The following  table  illustrates  how
       host  and  hostfile directives work together to define the set of hosts
       upon which a job	will execute in	the  absence  of  a  resource  manager

	hostfile      host	  hostfile	 Result
       ----------    ------	 ----------	 -----------------------------------------
	unset	     unset	    unset	 Job is	co-located with	mpirun
	unset	      set	    unset	 Host defines resource list for	the job
	unset	     unset	     set	 Hostfile defines resource list	for the	job
	unset	      set	     set	 Hostfile defines resource list	for the	job,
						 then host filters the list to define the final
						 set of	nodes available	to each	application
						 within	the job
	 set	     unset	    unset	 Default hostfile defines resource list	for the	job
	 set	      set	    unset	 Default hostfile defines resource list	for the	job,
						 then host filters the list to define the final
						 set of	nodes available	to each	application
						 within	the job
	 set	      set	     set	 Default hostfile defines resource list	for the	job,
						 then hostfile filters the list, and then host filters
						 the list to define the	final set of nodes available
						 to each application within the	job

       This  changes somewhat in the presence of a RM as that entity specifies
       the initial allocation of nodes.	In this	case,  the  default  hostfile,
       hostfile	and host directives are	all used to filter the RM's specifica-
       tion so that a user can utilize different portions  of  the  allocation
       for different jobs. This	is done	according to the same precedence order
       as in the prior table, with the RM providing the	initial	pool of	nodes.

       Once an initial allocation has been specified (whether by  an  RM,  de-
       fault  hostfile,	or hostfile), subsequent hostfile and -host specifica-
       tions can be made using relative	indexing. This allows a	user to	stipu-
       late  which hosts are to	be used	for a given app_context	without	speci-
       fying the particular host name, but rather its relative position	in the

       This can	probably best be understood through consideration of a few ex-
       amples. Consider	the case where an RM has allocated a set of  nodes  to
       the  user  named	 "foo1,	 foo2,	foo3,  foo4". The user wants the first
       app_context to have exclusive use of the	first two nodes, and a	second
       app_context to use the last two nodes. Of course, the user could	print-
       out the allocation to find the names of the nodes allocated to them and
       then use	-host to specify this layout, but this is cumbersome and would
       require hand-manipulation for every invocation.

       A simpler method	is to utilize OpenRTE's	relative  indexing  capability
       to specify the desired layout. In this case, a command line of:

       mpirun -pernode -host +n1,+n2 ./app1 : -host +n3,+n4 ./app2

       would  provide  the  desired pattern. The "+" syntax indicates that the
       information is being provided as	a relative index to the	existing allo-
       cation. Two methods of relative indexing	are supported:

       +n<#>  A	 relative  index into the allocation referencing the <#> node.
	      OpenRTE will substitute the <#> node in the allocation

	      A	request	for <#>	empty nodes - i.e., OpenRTE is	to  substitute
	      this reference with <#> nodes that have not yet been used	by any
	      other app_context. If the	":<#>" is not provided,	 OpenRTE  will
	      substitute the reference with all	empty nodes. Note that OpenRTE
	      does track the empty nodes that have been	assigned in this  man-
	      ner,  so	multiple uses of this option will result in assignment
	      of unique	nodes up to the	limit of the  available	 empty	nodes.
	      Requests	for  more empty	nodes than are available will generate
	      an error.

       Relative	indexing can be	combined with absolute naming of hosts in  any
       arbitrary  manner,  and	can  be	 used in hostfiles as well as with the
       -host command line option. In addition, any slot	specification provided
       in  hostfiles  will be respected	- thus,	a user can specify that	only a
       certain number of slots from a relative indexed host are	to be used for
       a given app_context.

       Another example may help	illustrate this	point. Consider	the case where
       a user has a default hostfile containing:

       dummy1 slots=4
       dummy2 slots=4
       dummy3 slots=4
       dummy4 slots=4
       dummy5 slots=4

       This may, for example, be a hostfile that describes a set of  commonly-
       used  resources	that  the user wishes to execute applications against.
       For this	particular application,	the user  plans	 to  map  byslot,  and
       wants  the  first two ranks to be on the	second node of any allocation,
       the next	ranks to land on an empty node,	have one rank specifically  on
       dummy4, the next	rank to	be on the second node of the allocation	again,
       and finally any remaining ranks to be on	whatever empty nodes are left.
       To accomplish this, the user provides a hostfile	of:

       +n2 slots=2
       dummy4 slots=1

       The user	can now	use this information in	combination with OpenRTE's se-
       quential	mapper to obtain their specific	layout:

       mpirun --default-hostfile dummyhosts -hostfile mylayout -mca rmaps seq ./my_app

       which will result in:

       rank0 being mapped to dummy3
       rank1 to	dummy1 as the first empty node
       rank2 to	dummy4
       rank3 to	dummy3
       rank4 to	dummy2 and rank5 to dummy5 as the last remaining unused	nodes

       Note that the sequential	mapper ignores the number of  slots  arguments
       as it only maps one rank	at a time to each node in the list.

       If the default round-robin mapper had been used,	then the mapping would
       have resulted in:

       ranks 0 and 1 being mapped to dummy3 since two slots were specified
       ranks 2-5 on dummy1 as the first	empty node, which has four slots
       rank6 on	dummy4 since the hostfile specifies only a single slot from that node is to be used
       ranks 7 and 8 on	dummy3 since only two slots remain available
       ranks 9-12 on dummy2 since it is	the next available empty node and has four slots
       ranks 13-16 on dummy5 since it is the last remaining unused node	and has	four slots

       Thus, the use of	relative indexing can allow for	complex	mappings to be
       ported  across allocations, including those obtained from automated re-
       source managers,	without	the need for manual  manipulation  of  scripts
       and/or command lines.


4.1.3				 Mar 31, 2022			 ORTE_HOSTS(7)


Want to link to this manual page? Use this URL:

home | help