CODEWITHSUNDEEP
dpdkvirtio
Tuesday
Tuesday

Reputation: 21

In vhost-user, how does the feature VHOST_USER_PROTOCOL_F_HOST_NOTIFIER work?

I am researching the code for the mlx5 vdpa driver, and this part of the code registers the callback function mlx5_vdpa_virtq_kick_handler for kickfd:

/* Setup doorbell mapping. */
virtq->intr_handle.fd = vq.kickfd;
if (virtq->intr_handle.fd == -1) {
    DRV_LOG(WARNING, "Virtq %d kickfd is invalid.", index);
} else {
    virtq->intr_handle.type = RTE_INTR_HANDLE_EXT;
    if (rte_intr_callback_register(&virtq->intr_handle,
                        mlx5_vdpa_virtq_kick_handler,
                        virtq)) {
        virtq->intr_handle.fd = -1;
        DRV_LOG(ERR, "Failed to register virtq %d interrupt.",
            index);
        goto error;
    } else {
        DRV_LOG(DEBUG, "Register fd %d interrupt for virtq %d.",
            virtq->intr_handle.fd, index);
    }
}

In this callback function, the rte_vhost_host_notifier_ctrl function is called, in which VhostUserMsg of type VHOST_USER_SLAVE_VRING_HOST_NOTIFIER_MSG is constructed and sent to qemu (the simulator I use).

struct VhostUserMsg msg = {
    .request.slave = VHOST_USER_SLAVE_VRING_HOST_NOTIFIER_MSG,
    .flags = VHOST_USER_VERSION | VHOST_USER_NEED_REPLY,
    .size = sizeof(msg.payload.area),
    .payload.area = {
        .u64 = index & VHOST_USER_VRING_IDX_MASK,
        .size = size,
        .offset = offset,
    },
};

if (fd < 0)
    msg.payload.area.u64 |= VHOST_USER_VRING_NOFD_MASK;
else {
    msg.fds[0] = fd;
    msg.fd_num = 1;
}

I have a general understanding of the functionality of this code, but I am not sure how it is implemented? Because from the print information of the mlx5 vdpa driver, it appears that the callback function for this kickfd has only been triggered once. Where did the subsequent "kick" go?

This patch introduces VHOST_USER_PROTOCOL_F_HOST_NOTIFIER.
With this feature negotiated, vhost-user backend can register
memory region based host notifiers. And it will allow the guest
driver in the VM to notify the hardware accelerator at the
vhost-user backend directly.

The doorbell message was only printed once per-queue:

mlx5_vdpa: vid 0: Init last_avail_idx=0, last_used_idx=0 for virtq 0.
mlx5_vdpa: Register fd 76 interrupt for virtq 0.
mlx5_vdpa: vid 0 virtq 0 was created successfully.
mlx5_vdpa: Virtq 0 notifier state is enabled.
mlx5_vdpa: Ring virtq 0 doorbell.
mlx5_vdpa: vid 0: Init last_avail_idx=0, last_used_idx=0 for virtq 1.
mlx5_vdpa: Register fd 81 interrupt for virtq 1.
mlx5_vdpa: vid 0 virtq 1 was created successfully.
mlx5_vdpa: Virtq 1 notifier state is enabled.
mlx5_vdpa: Ring virtq 1 doorbell.
mlx5_vdpa: vDPA device 0 was configured.

Upvotes: 0

Views: 83

Answers (1)

artisignal
artisignal

Reputation: 401

By the looks of it, two modes for the feature in question might exist: kick and notification area (also known as passthrough). The latter allows to set up direct doorbell mapping for the guest. Based on that, it may be support for passthrough mode being discovered that causes subsequent kick generation to cease.

For more information, one can refer to the specific commit that added support for the feature to the vDPA driver in question. It is also possible to search for similar keywords within the Linux kernel.

Upvotes: 0

Related Questions