Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages

  
 
  

home | help
fi_rma(3)		       Libfabric v1.14.0		     fi_rma(3)

NAME
       fi_rma -	Remote memory access operations

       fi_read / fi_readv / fi_readmsg
	      Initiates	a read from remote memory

       fi_write	 /  fi_writev  /  fi_writemsg fi_inject_write /	fi_writedata :
       Initiate	a write	to remote memory

SYNOPSIS
	      #include <rdma/fi_rma.h>

	      ssize_t fi_read(struct fid_ep *ep, void *buf, size_t len,	void *desc,
		  fi_addr_t src_addr, uint64_t addr, uint64_t key, void	*context);

	      ssize_t fi_readv(struct fid_ep *ep, const	struct iovec *iov, void	**desc,
		  size_t count,	fi_addr_t src_addr, uint64_t addr, uint64_t key,
		  void *context);

	      ssize_t fi_readmsg(struct	fid_ep *ep, const struct fi_msg_rma *msg,
		  uint64_t flags);

	      ssize_t fi_write(struct fid_ep *ep, const	void *buf, size_t len,
		  void *desc, fi_addr_t	dest_addr, uint64_t addr, uint64_t key,
		  void *context);

	      ssize_t fi_writev(struct fid_ep *ep, const struct	iovec *iov, void **desc,
		  size_t count,	fi_addr_t dest_addr, uint64_t addr, uint64_t key,
		  void *context);

	      ssize_t fi_writemsg(struct fid_ep	*ep, const struct fi_msg_rma *msg,
		  uint64_t flags);

	      ssize_t fi_inject_write(struct fid_ep *ep, const void *buf, size_t len,
		  fi_addr_t dest_addr, uint64_t	addr, uint64_t key);

	      ssize_t fi_writedata(struct fid_ep *ep, const void *buf, size_t len,
		  void *desc, uint64_t data, fi_addr_t dest_addr, uint64_t addr,
		  uint64_t key,	void *context);

	      ssize_t fi_inject_writedata(struct fid_ep	*ep, const void	*buf, size_t len,
		  uint64_t data, fi_addr_t dest_addr, uint64_t addr, uint64_t key);

ARGUMENTS
       ep     Fabric endpoint on which to initiate read	or write operation.

       buf    Local data buffer	to read	 into  (read  target)  or  write  from
	      (write source)

       len    Length  of  data	to  read  or write, specified in bytes.	 Valid
	      transfers	are from 0 bytes up to the endpoint's max_msg_size.

       iov    Vectored data buffer.

       count  Count of vectored	data entries.

       addr   Address of remote	memory to access.  This	will  be  the  virtual
	      address  of the remote region in the case	of FI_MR_BASIC,	or the
	      offset from the starting address in the case of FI_MR_SCALABLE.

       key    Protection key associated	with the remote	memory.

       desc   Descriptor associated with the local data	buffer See fi_mr(3).

       data   Remote CQ	data to	transfer with the operation.

       dest_addr
	      Destination address for connectionless write transfers.  Ignored
	      for connected endpoints.

       src_addr
	      Source  address  to read from for	connectionless transfers.  Ig-
	      nored for	connected endpoints.

       msg    Message descriptor for read and write operations.

       flags  Additional flags to apply	for the	read or	write operation.

       context
	      User specified pointer to	associate with	the  operation.	  This
	      parameter	 is  ignored if	the operation will not generate	a suc-
	      cessful completion, unless an op flag specifies the context  pa-
	      rameter be used for required input.

DESCRIPTION
       RMA (remote memory access) operations are used to transfer data direct-
       ly between a local data buffer and a remote data	buffer.	 RMA transfers
       occur  on a byte	level granularity, and no message boundaries are main-
       tained.

       The  write  functions  -	 fi_write,  fi_writev,	 fi_writemsg,	fi_in-
       ject_write,  and	fi_writedata - are used	to transmit data into a	remote
       memory buffer.  The main	difference between  write  functions  are  the
       number  and  type  of parameters	that they accept as input.  Otherwise,
       they perform the	same general function.

       The read	functions - fi_read, fi_readv, and fi_readmsg -	 are  used  to
       transfer	 data  from  a remote memory region into local data buffer(s).
       Similar to the write operations,	read operations	operate	 asynchronous-
       ly.   Users  should  not	touch the posted data buffer(s)	until the read
       operation has completed.

       Completed RMA operations	are reported to	the user through one  or  more
       completion  queues associated with the endpoint.	 Users provide context
       which are associated with each operation, and is	returned to  the  user
       as part of the completion.  See fi_cq for completion event details.

       By  default,  the  remote endpoint does not generate an event or	notify
       the user	when a memory region has been accessed by an RMA read or write
       operation.  However, immediate data may be associated with an RMA write
       operation.  RMA writes with immediate data will generate	 a  completion
       entry  at the remote endpoint, so that the immediate data may be	deliv-
       ered.

   fi_write
       The call	fi_write transfers the data contained  in  the	user-specified
       data buffer to a	remote memory region.

   fi_writev
       The  fi_writev call adds	support	for a scatter-gather list to fi_write.
       The fi_writev transfers the set of data buffers referenced by  the  iov
       parameter to the	remote memory region.

   fi_writemsg
       The  fi_writemsg	 call  supports	data transfers over both connected and
       connectionless endpoints, with the ability to control the write	opera-
       tion per	call through the use of	flags.	The fi_writemsg	function takes
       a struct	fi_msg_rma as input.

	      struct fi_msg_rma	{
		  const	struct iovec *msg_iov;	   /* local scatter-gather array */
		  void		     **desc;	   /* operation	descriptor */
		  size_t	     iov_count;	   /* #	elements in msg_iov */
		  fi_addr_t	     addr;	  /* optional endpoint address */
		  const	struct fi_rma_iov *rma_iov;/* remote SGL */
		  size_t	     rma_iov_count;/* #	elements in rma_iov */
		  void		     *context;	   /* user-defined context */
		  uint64_t	     data;	   /* optional immediate data */
	      };

	      struct fi_rma_iov	{
		  uint64_t	     addr;	   /* target RMA address */
		  size_t	     len;	   /* size of target buffer */
		  uint64_t	     key;	   /* access key */
	      };

   fi_inject_write
       The write inject	call is	an optimized version of	fi_write.  It provides
       similar completion semantics as fi_inject fi_msg(3).

   fi_writedata
       The  write data call is similar to fi_write, but	allows for the sending
       of remote CQ data (see FI_REMOTE_CQ_DATA	flag) as part of the transfer.

   fi_inject_writedata
       The inject write	data call is similar to	 fi_inject_write,  but	allows
       for  the	sending	of remote CQ data (see FI_REMOTE_CQ_DATA flag) as part
       of the transfer.

   fi_read
       The fi_read call	requests that the remote endpoint transfer  data  from
       the remote memory region	into the local data buffer.

   fi_readv
       The  fi_readv  call  adds support for a scatter-gather list to fi_read.
       The fi_readv transfers data from	the remote memory region into the  set
       of data buffers referenced by the iov parameter.

   fi_readmsg
       The  fi_readmsg	call  supports	data transfers over both connected and
       connectionless endpoints, with the ability to control the  read	opera-
       tion  per call through the use of flags.	 The fi_readmsg	function takes
       a struct	fi_msg_rma as input.

FLAGS
       The fi_readmsg and fi_writemsg calls allow the user  to	specify	 flags
       which  can change the default data transfer operation.  Flags specified
       with fi_readmsg / fi_writemsg override most flags previously configured
       with the	endpoint, except where noted (see fi_endpoint.3).  The follow-
       ing list	of flags are usable with fi_readmsg and/or fi_writemsg.

       FI_REMOTE_CQ_DATA
	      Applies to fi_writemsg and fi_writedata.	Indicates that	remote
	      CQ  data is available and	should be sent as part of the request.
	      See fi_getinfo for additional details on FI_REMOTE_CQ_DATA.

       FI_COMPLETION
	      Indicates	that a completion entry	should be  generated  for  the
	      specified	operation.  The	endpoint must be bound to a completion
	      queue with FI_SELECTIVE_COMPLETION that corresponds to the spec-
	      ified operation, or this flag is ignored.

       FI_MORE
	      Indicates	 that the user has additional requests that will imme-
	      diately be posted	after the current call returns.	 Use  of  this
	      flag  may	 improve performance by	enabling the provider to opti-
	      mize its access to the fabric hardware.

       FI_INJECT
	      Applies to fi_writemsg.  Indicates that the outbound data	buffer
	      should  be returned to user immediately after the	write call re-
	      turns, even if the operation is  handled	asynchronously.	  This
	      may require that the underlying provider implementation copy the
	      data into	a local	buffer and transfer out	of that	buffer.	  This
	      flag can only be used with messages smaller than inject_size.

       FI_INJECT_COMPLETE
	      Applies  to  fi_writemsg.	 Indicates that	a completion should be
	      generated	when the source	buffer(s) may be reused.

       FI_TRANSMIT_COMPLETE
	      Applies to fi_writemsg.  Indicates that a	completion should  not
	      be generated until the operation has been	successfully transmit-
	      ted and is no longer being tracked by the	provider.

       FI_DELIVERY_COMPLETE
	      Applies to fi_writemsg.  Indicates that a	completion  should  be
	      generated	 when the operation has	been processed by the destina-
	      tion.

       FI_COMMIT_COMPLETE
	      Applies to fi_writemsg when targeting persistent memory regions.
	      Indicates	 that  a completion should be generated	only after the
	      result of	the operation has been made durable.

       FI_FENCE
	      Applies to transmits.  Indicates that the	 requested  operation,
	      also known as the	fenced operation, and any operation posted af-
	      ter the fenced operation will be deferred	until all previous op-
	      erations targeting the same peer endpoint	have completed.	 Oper-
	      ations posted after the fencing will see and/or replace the  re-
	      sults of any operations initiated	prior to the fenced operation.

       The ordering of operations starting at the posting of the fenced	opera-
       tion (inclusive)	to the posting of a subsequent fenced  operation  (ex-
       clusive)	is controlled by the endpoint's	ordering semantics.

RETURN VALUE
       Returns 0 on success.  On error,	a negative value corresponding to fab-
       ric errno is returned.  Fabric errno values are defined in  rdma/fi_er-
       rno.h.

ERRORS
       -FI_EAGAIN
	      See fi_msg(3) for	a detailed description of handling FI_EAGAIN.

SEE ALSO
       fi_getinfo(3), fi_endpoint(3), fi_domain(3), fi_cq(3)

AUTHORS
       OpenFabrics.

Libfabric Programmer's Manual	  2021-03-22			     fi_rma(3)

NAME | SYNOPSIS | ARGUMENTS | DESCRIPTION | FLAGS | RETURN VALUE | ERRORS | SEE ALSO | AUTHORS

Want to link to this manual page? Use this URL:
<https://www.freebsd.org/cgi/man.cgi?query=fi_rma&sektion=3&manpath=FreeBSD+13.1-RELEASE+and+Ports>

home | help