Reputation: 1
I am writing an application over DPDK that must handle high traffic throughput. I need shared references to the same Mbuf, and I would like to use DPDK's built-in rte_mbuf_refcnt_update
for performance.
When I try to do this, the application mempool is quickly exhausted. I looked into it, and it seems like the rte_mbuf_refcnt_update
doesn't release the mbuf. However, I could imagine a race condition if I did a "read-then-free"?
Here's a basic recreation of the relevant logic (Rust):
fn new(mbuf: *mut dpdk::rte_mbuf) -> Self {
assert!(unsafe { dpdk::rte_mbuf_refcnt_read(self.raw()) } == 1);
Self { raw: NonNull::new_unchecked(mbuf) }
}
fn ref(mbuf: *mut dpdk::rte_mbuf) -> Self {
unsafe { dpdk::rte_mbuf_refcnt_update(mbuf, 1); }
Self { raw: NonNull::new_unchecked(mbuf) }
}
// ...
// impl Drop
// Option 1: just decrement refcnt
fn drop(&mut self) {
unsafe { dpdk::rte_mbuf_refcnt_update(self.raw_mut(), -1); }
// Would this also free the Mbuf?
}
// Alternative: if refcount == 1, free
// impl Drop
fn drop(&mut self) {
if unsafe { dpdk::rte_mbuf_refcnt_update(self.raw_mut(), -1) } == 0 {
unsafe { dpdk::rte_mbuf_refcnt_set(self.raw_mut(), 1) };
// Race here - two threads invoke `rte_pktmbuf_free` -->
// double free or invalid access? Undefined behavior if mbuf
// re-allocated?
unsafe { dpdk::rte_pktmbuf_free(self.raw()) };
}
}
EDIT: Based on the responses, it looks like the following would be reasonable:
fn drop(...) {
// This will decrement internally, but not free unless
// no out-standing references
unsafe { dpdk::rte_pktmbuf_free(self.raw()) };
}
Alternatively, the second alternative above in my original post should have a similar outcome. And maybe could be more efficient to avoid some of the extra logic in rte_pktmbuf_free
(not sure)?
However, I am still concerned about undefined behavior in the event of multiple threads invoking free
concurrently. I assume I'm missing something in the source code that protects against this.
Does this seem reasonable?
TL;DR - what's the correct way to use dpdk reference counting?
I'm using DPDK 21.08 but could upgrade.
Thank you so much!
Upvotes: 0
Views: 62