Skip to content

Consolidate duplicated reactor logic across epoll, kqueue, and select backends #192

@sgerbino

Description

@sgerbino

Summary

The epoll, kqueue, and select backends share large amounts of duplicated code. A common "reactor" abstraction could consolidate this while keeping backend-specific details (syscalls, event structures, timer/interrupt mechanisms) isolated.

Duplicated code (high priority)

  1. Base op struct (epoll_op, kqueue_op, select_op): fields, reset(), destroy(), request_cancel(), complete(), start(), canceller — all identical. Only perform_io() bodies differ. Select's registered tri-state atomic should be replaced with the descriptor_state pattern used by epoll and kqueue.

  2. Socket/acceptor impl classes (*_socket.hpp, *_acceptor.hpp): the virtual interface, embedded op slots, and member layout are byte-for-byte identical across all three backends.

  3. Service state classes (*_socket_state, *_acceptor_state): identical layout, just different type names — could be a single template.

  4. cancel() and cancel_single_op() in epoll and kqueue socket services: word-for-word identical (~60 lines each).

  5. Scheduler threading machinery (epoll and kqueue): scheduler_context, thread_context_guard, find_context, all of post(), run(), run_one(), poll(), poll_one(), work_started(), work_finished(), drain_thread_queue(), post_deferred_completions(), full signal state machine — ~400 lines of identical code.

  6. read_some() / write_some() dispatch in epoll and kqueue services: the three-phase pattern (speculative I/O, inline budget, park in reactor) is identical; only the syscall differs. Select should adopt this pattern once it uses descriptor_state.

Prerequisite: migrate select to descriptor_state

The select backend currently uses a select_registration_state tri-state atomic per-op for fd registration. This should be replaced with the descriptor_state pattern that epoll and kqueue share, which would:

  • Unify the op struct across all three backends (no select-only registered field)
  • Allow select to share the same cancel(), cancel_single_op(), and register_op() logic
  • Enable the speculative I/O + inline budget path for select

Legitimately different (should stay separate)

  • run_task() / run_reactor(): epoll_wait vs kevent vs select have fundamentally different event structures
  • open_socket(): SOCK_NONBLOCK|SOCK_CLOEXEC single-call (Linux) vs fcntl sequence (BSD/POSIX)
  • Timer wakeup: timerfd (epoll), EVFILT_TIMER/timeout (kqueue), timeval timeout (select)
  • Reactor interruption: eventfd (epoll), self-pipe/EVFILT_USER (kqueue), pipe (select)
  • Write syscall: sendmsg+MSG_NOSIGNAL (epoll/select) vs writev with SO_NOSIGPIPE (kqueue)

Suggested approach

Introduce a common reactor_ naming convention with shared base/template classes parameterized on the backend. Backend-specific code (syscalls, event dispatch, timer/interrupt mechanisms) stays in per-backend files.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions