librpma API version 0.11.0

librpma.7 rpma_conn_apply_remote_peer_cfg.3 rpma_conn_cfg_delete.3 rpma_conn_cfg_get_cq_size.3 rpma_conn_cfg_get_rcq_size.3 rpma_conn_cfg_get_rq_size.3 rpma_conn_cfg_get_sq_size.3 rpma_conn_cfg_get_timeout.3 rpma_conn_cfg_new.3 rpma_conn_cfg_set_cq_size.3 rpma_conn_cfg_set_rcq_size.3 rpma_conn_cfg_set_rq_size.3 rpma_conn_cfg_set_sq_size.3 rpma_conn_cfg_set_timeout.3 rpma_conn_delete.3 rpma_conn_disconnect.3 rpma_conn_get_cq.3 rpma_conn_get_event_fd.3 rpma_conn_get_private_data.3 rpma_conn_get_qp_num.3 rpma_conn_get_rcq.3 rpma_conn_next_event.3 rpma_conn_req_connect.3 rpma_conn_req_delete.3 rpma_conn_req_get_private_data.3 rpma_conn_req_new.3 rpma_conn_req_recv.3 rpma_cq_get_completion.3 rpma_cq_get_fd.3 rpma_cq_get_wc.3 rpma_cq_wait.3 rpma_ep_get_fd.3 rpma_ep_listen.3 rpma_ep_next_conn_req.3 rpma_ep_shutdown.3 rpma_err_2str.3 rpma_flush.3 rpma_log_get_threshold.3 rpma_log_set_function.3 rpma_log_set_threshold.3 rpma_mr_advise.3 rpma_mr_dereg.3 rpma_mr_get_descriptor.3 rpma_mr_get_descriptor_size.3 rpma_mr_get_ptr.3 rpma_mr_get_size.3 rpma_mr_reg.3 rpma_mr_remote_delete.3 rpma_mr_remote_from_descriptor.3 rpma_mr_remote_get_flush_type.3 rpma_mr_remote_get_size.3 rpma_peer_cfg_delete.3 rpma_peer_cfg_from_descriptor.3 rpma_peer_cfg_get_descriptor.3 rpma_peer_cfg_get_descriptor_size.3 rpma_peer_cfg_get_direct_write_to_pmem.3 rpma_peer_cfg_new.3 rpma_peer_cfg_set_direct_write_to_pmem.3 rpma_peer_delete.3 rpma_peer_new.3 rpma_read.3 rpma_recv.3 rpma_send.3 rpma_send_with_imm.3 rpma_utils_conn_event_2str.3 rpma_utils_get_ibv_context.3 rpma_utils_ibv_context_is_odp_capable.3 rpma_write.3 rpma_write_atomic.3 rpma_write_with_imm.3


rpma_cq_get_completion - receive a completion of an operation (deprecated)


      #include <librpma.h>

      struct rpma_cq;
      enum rpma_op {
      struct rpma_completion {
              void *op_context;
              enum rpma_op op;
              uint32_t byte_len;
              enum ibv_wc_status op_status;
              unsigned flags;
              uint32_t imm;

      int rpma_cq_get_completion(struct rpma_cq *cq,
                      struct rpma_completion *cmpl);


rpma_cq_get_completion() receives the next available completion of an already posted operation. All operations generate completion on error. The operations posted with the RPMA_F_COMPLETION_ALWAYS flag also generate a completion on success.

The rpma_completion structure provides the following fields:

  • op_context - context of the operation provided by the user to either rpma_conn_req_recv(3), rpma_flush(3), rpma_read(3), rpma_recv(3), rpma_send(3), rpma_send_with_imm(3), rpma_write(3), rpma_write_atomic(3), rpma_write_with_imm(3)

  • op - type of the operation, for available values please see the description below

  • byte_len - number of bytes transferred

  • op_status - status of the operation

  • flags - flags of the operation, for available values please see ibv_poll_cq(3)

  • imm - immediate data (in host byte order)

The available op values are:

  • RPMA_OP_READ - RMA read operation

  • RPMA_OP_WRITE - RMA write operation

  • RPMA_OP_FLUSH - RMA flush operation

  • RPMA_OP_SEND - messaging send operation

  • RPMA_OP_RECV - messaging receive operation

  • RPMA_OP_RECV_RDMA_WITH_IMM - messaging receive operation for RMA write operation with immediate data

Note that if the provided cq is the main CQ and the receive CQ is present on the same connection this function won't return RPMA_OP_RECV and RPMA_OP_RECV_RDMA_WITH_IMM at any time. The receive CQ has to be used instead to collect these completions. Please see the rpma_conn_get_rcq(3) for details about the receive CQ.


The rpma_cq_get_completion() function returns 0 on success or a negative error code on failure. On success, it writes the first available completion to cmpl. If op_status of the written cmpl is not equal to IBV_WC_SUCCESS then only op_context of the returned cmpl is valid.


rpma_cq_get_completion() can fail with the following errors:

  • RPMA_E_INVAL - cq or cmpl is NULL

  • RPMA_E_NO_COMPLETION - no completions available

  • RPMA_E_PROVIDER - ibv_poll_cq(3) failed with a provider error

  • RPMA_E_UNKNOWN - ibv_poll_cq(3) failed but no provider error is available

  • RPMA_E_NOSUPP - not supported opcode


This is an example snippet of code using the old API:

        struct rpma_completion cmpl;

        ret = rpma_cq_get_completion(cq, &cmpl);
        if (ret) { error_handling_code() }

        if (cmpl.op_status != IBV_WC_SUCCESS) { error_handling_code() }
        if (cmpl.op != RPMA_OP_READ) { error_handling_code() }

        void *op_context = cmpl.op_context;
        uint32_t byte_len = cmpl.byte_len;
        enum ibv_wc_status op_status = cmpl.op_status;
        unsigned flags = cmpl.flags;
        uint32_t imm = cmpl.imm;

The above snippet should be replaced with the following one using the new API:

        struct ibv_wc wc;

        ret = rpma_cq_get_wc(cq, 1, &wc, NULL);
        if (ret) { error_handling_code() }

        if (wc.status != IBV_WC_SUCCESS) { error_handling_code() }
        if (wc.opcode != IBV_WC_RDMA_READ) { error_handling_code() }

        void *op_context = (void *)wc.wr_id;
        uint32_t byte_len = wc.byte_len;
        enum ibv_wc_status op_status = wc.status;
        unsigned flags = (unsigned)wc.wc_flags;
        uint32_t imm = ntohl(wc.imm_data);


rpma_conn_get_cq(3), rpma_conn_get_rcq(3), rpma_conn_req_recv(3), rpma_cq_wait(3), rpma_cq_get_fd(3), rpma_flush(3), rpma_read(3), rpma_recv(3), rpma_send(3), rpma_send_with_imm(3), rpma_write(3), rpma_write_atomic(3), rpma_write_with_imm(3), librpma(7) and

The contents of this web site and the associated GitHub repositories are BSD-licensed open source.