flare.runtime.uring_reactor

Set up the ring + (optionally) the wakeup eventfd.

Free the wakeup buffer + close the wakeup fd; the IoUringDriver destructor handles ring teardown.

Return the underlying io_uring ring fd.

Kernel-allocated SQ size.

Kernel-allocated CQ size.

Submit one multishot accept SQE so the kernel posts a CQE for every accepted connection without re-arming.

The CQE handler should look at cqe.has_more — when unset, the multishot has terminated (e.g. listener closed) and the caller should re-arm.

Submit one recv SQE that re-fires once when data arrives.

NOTE on naming: the original intent was true multishot (IORING_RECV_MULTISHOT) but the Linux kernel rejects multishot recv with -EINVAL unless IOSQE_BUFFER_SELECT is set on the SQE (kernel needs a buffer pool to pick from on each completion -- it can't pre-allocate space for an unbounded stream into a single fixed buffer). This helper targets a single user-owned buffer, so it's effectively one-shot at the kernel level. For TRUE multishot recv, use :func:arm_recv_buffer_select with a registered IORING_REGISTER_PBUF_RING buffer ring.

Submit one IORING_OP_PROVIDE_BUFFERS SQE that hands the kernel a contiguous run of nbufs buffers of nbytes_per_buf bytes each, starting at addr with ids [bid, bid + nbufs) in buffer-group bgid.

After this SQE completes (one CQE tagged URING_OP_PROVIDE_BUFFERS with conn_id = bgid and res = number of buffers actually accepted), subsequent arm_recv_buffer_select calls with the same bgid will have the kernel auto-pick a free buffer for each recv.

flare's recv-buffer-ring dispatch loop typically calls this once at startup with N×8 KiB buffers per worker, then re-arms the same buffer id after each recv CQE is processed (one PROVIDE_BUFFERS SQE with nbufs=1 per recv CQE -- a cheap re-fill that amortises into the next io_uring_enter).

Register a kernel-mapped provided-buffer ring of ring_entries slots in buffer-group bgid via IORING_REGISTER_PBUF_RING (kernel 5.19+).

The 2.7x faster successor to :func:arm_provide_buffers per Linux kernel benchmarks. Once registered, the ring is populated by writing struct io_uring_buf entries and bumping the tail pointer in shared memory -- NO SQE per refill, NO io_uring_enter per refill. The kernel reads the tail with acquire semantics on every recv that selects from this group, so applications can replenish single buffers cheaply.

Returns the ring memory address. Caller is responsible for :func:pbuf_ring_add_buffer + :func:pbuf_ring_advance to seed the ring; flare's bufring dispatch typically does this once with all N buffers at startup, then re-adds one buffer per recv CQE before the next io_uring_enter.

Drop a ring previously registered via :func:register_pbuf_ring. Kernel side cleans up its reference; userspace side munmaps the ring memory.

Submit one IORING_OP_RECV SQE with IOSQE_BUFFER_SELECT set; the kernel picks a buffer from the bgid pool at recv time.

This is the production HTTP server recv shape every Rust io_uring HTTP server uses. Combined with multishot=True (default), one SQE per accepted connection drives an unbounded stream of recv CQEs; each CQE points at a fresh kernel-picked buffer (id in the high 16 bits of UringCompletion.flags, decoded via :func:IoUringCqe.buffer_id). No per-conn buffer ownership, no per-CQE re-arm, no recv syscall round-trip.

Submit one IORING_OP_SEND SQE.

Submit one IORING_OP_CLOSE SQE.

Uses IOSQE_CQE_SKIP_SUCCESS so the kernel only posts a CQE on failure — the typical close path is async fire-and-forget.

Submit a multishot IORING_OP_POLL_ADD against fd.

Kernel posts a CQE every time the fd's readiness matches any bit in poll_mask, without re-arming. This is the io_uring analog of epoll_ctl(EPOLL_CTL_ADD, fd, EPOLLIN | EPOLLET) and is the substrate the upcoming server-loop dispatch swap (B0 wire-in) uses to replace epoll_wait on the io_uring backend.

The CQE is tagged URING_OP_POLL so the dispatch loop can route it to the same code path that handles EVENT_READABLE on the epoll backend; the existing ConnHandle.on_readable then runs its own recv syscall. The buffer-ring path (IORING_OP_RECV + IORING_RECV_MULTISHOT against a registered IORING_REGISTER_PBUF_RING) is a separate dispatch loop opt-in via FLARE_BUFRING_HANDLER=1.

Submit IORING_OP_POLL_REMOVE cancelling the multishot poll registered for conn_id.

After the kernel processes the remove SQE, two CQEs arrive: one tagged URING_OP_POLL_REMOVE (this SQE's own completion, res = 0 on success or -ENOENT if nothing matched), and one final tagged URING_OP_POLL without IORING_CQE_F_MORE for the cancelled poll itself.

Submit an IORING_OP_ASYNC_CANCEL for any in-flight op tagged with conn_id (the kernel cancels the first match).

Submit any pending SQEs, drain ready CQEs, then optionally block for more.

Peek-then-block dispatch (matches the tokio-uring / monoio pattern). Three phases per call:

1. Submit: flush any pending SQEs to the kernel via io_uring_enter(submit, 0, GETEVENTS) -- this does NOT block, but does collect any CQEs the kernel has already posted asynchronously (e.g. multishot recv completions that fired between the prior poll and this one). On kernels with DEFER_TASKRUN (negotiated by the bufring setup-flag probe), this call also runs deferred task work which can produce the CQEs we then drain in step 2.

2. Drain: pull every ready CQE out of the CQ into out, up to max_completions. No syscall in this phase -- pure userspace ring head/tail manipulation. This is where multishot recv CQEs land in batches under sustained load; an always-block design would only pull one at a time because submit_and_wait(1) returns as soon as one CQE is ready.

3. Block (only if needed): if we drained fewer than min_complete CQEs, call submit_and_wait(...) with the remaining count to actually block. The common case under sustained load is that step 2 returned plenty of CQEs and step 3 is skipped entirely.

Returns the number of completions appended to out.

Cross-thread wakeup: write 1 to the eventfd. Safe from any thread.

The next poll will return because the multishot recv on the eventfd posts a CQE.

No-op when the reactor was constructed with enable_wakeup=False -- single-issuer bufring rings opt out because they don't need cross-thread signalling.