Skip to content

Add awakened_from_remote() to bypass remote_ready_queue - #3

Open
Akatsukis wants to merge 1 commit into
categoryfrom
xiaojun/awakened_from_remote_v2
Open

Add awakened_from_remote() to bypass remote_ready_queue#3
Akatsukis wants to merge 1 commit into
categoryfrom
xiaojun/awakened_from_remote_v2

Conversation

@Akatsukis

Copy link
Copy Markdown
Collaborator

Depends on #2

@Akatsukis Akatsukis changed the title Xiaojun/awakened from remote v2 Add awakened_from_remote() to bypass remote_ready_queue May 13, 2026
@Akatsukis
Akatsukis force-pushed the xiaojun/awakened_from_remote_v2 branch from c81895e to 0f0d947 Compare May 13, 2026 16:12
@Akatsukis
Akatsukis requested a review from Copilot May 13, 2026 16:16

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds an optional awakened_from_remote() hook on algorithm so that schedulers with thread-safe ready-queues can bypass the per-thread remote_ready_queue_ when schedule_from_remote() is invoked. It also (apparently inheriting from a dependent PR) removes the sleep-queue mechanism and all timed-wait/sleep public API along with associated tests and examples, and replaces algorithm::suspend_until with a parameterless suspend().

Changes:

  • Add algorithm::awakened_from_remote() virtual hook (default returns false) and integrate fast-path in scheduler::schedule_from_remote, plus a new worker_splk_ to serialize attach/detach of worker contexts across threads.
  • Remove the sleep queue and all wait_until/wait_for/sleep_for/sleep_until API surface from scheduler, context, condition_variable[_any], future/shared_future, buffered_channel, operations, and wait_queue.
  • Delete the corresponding _post/_dispatch test files and the priority/wait_stuff examples, with matching removals in test/Jamfile.v2 and examples/Jamfile.v2.

Reviewed changes

Copilot reviewed 32 out of 32 changed files in this pull request and generated no comments.

Show a summary per file
File Description
include/boost/fiber/algo/algorithm.hpp Replace suspend_until with suspend; add awakened_from_remote default hook.
include/boost/fiber/algo/round_robin.hpp Override suspend() instead of suspend_until.
src/algo/round_robin.cpp Implementation now waits unconditionally on the cv.
include/boost/fiber/scheduler.hpp Remove sleep-queue type/state and wait_until; add worker_splk_ and AWAKENED_FROM_REMOTE guards.
src/scheduler.cpp Remove sleep2ready_/wait_until; integrate awakened_from_remote fast path; lock worker_splk_ in attach/detach/terminate/dispatch.
include/boost/fiber/context.hpp Drop sleep_hook, sleep_waker_, tp_, sleep_*/wait_until members.
src/context.cpp Remove wait_until and sleep_* definitions/asserts.
include/boost/fiber/waker.hpp, src/waker.cpp Remove suspend_and_wait_until.
include/boost/fiber/condition_variable.hpp Remove wait_for/wait_until overloads on cv and cv_any.
include/boost/fiber/buffered_channel.hpp Remove push_wait_for/until and pop_wait_for/until.
include/boost/fiber/future/future.hpp, .../shared_state.hpp Remove wait_for/wait_until.
include/boost/fiber/operations.hpp Remove sleep_for/sleep_until.
test/Jamfile.v2 Drop test entries for the removed _post/_dispatch test files.
test/test_fiber_{post,dispatch}.cpp, test/test_mutex_, test/test_condition_variable*, test/test_future, test/test_shared_future_, test/test_buffered_channel_* Deleted.
examples/Jamfile.v2, examples/priority.cpp Remove priority and wait_stuff examples.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@Akatsukis
Akatsukis requested review from jhunsaker and magedm May 13, 2026 16:19
@jhunsaker
jhunsaker force-pushed the category branch 2 times, most recently from 133ac78 to a7c0273 Compare May 29, 2026 05:45
@jhunsaker
jhunsaker force-pushed the category branch 2 times, most recently from fee1eda to dbab7e1 Compare August 3, 2026 18:45
Introduces an optional algorithm hook, awakened_from_remote(context*),
that lets a scheduling algorithm with a thread-safe ready-queue (e.g.
a shared priority queue) take ownership of a remote wakeup directly,
bypassing the per-thread remote_ready_queue_ + spinlock indirection.

When BOOST_FIBERS_AWAKENED_FROM_REMOTE is defined:
  - schedule_from_remote() offers non-pinned contexts to the algorithm
    first; on a true return the wake is claimed and the algorithm is
    notified. Falls back to remote_ready_queue_ for pinned contexts or
    when the algorithm declines.
  - A new worker_splk_ serializes attach_worker_context() and
    detach_worker_context() against each other and against
    worker_queue_ traversal in dispatch()/terminate(), so a remote
    thread that calls ctx->detach() inside awakened_from_remote()
    cannot race the owning thread attaching another fiber.
  - A compile-time guard rejects builds that combine the new hook
    with BOOST_FIBERS_NO_ATOMICS (single-thread mode).

Invariant asserts in schedule_from_remote() are hoisted above the
fast-path so the debug-time contract still applies when the hook
claims the wake.

Default awakened_from_remote() returns false, so algorithms that
don't override it behave exactly as before.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@jhunsaker
jhunsaker force-pushed the xiaojun/awakened_from_remote_v2 branch from 0f0d947 to 041e14a Compare August 3, 2026 21:31
Copilot AI review requested due to automatic review settings August 3, 2026 21:31

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (3)

include/boost/fiber/scheduler.hpp:86

  • worker_splk_ is currently always part of scheduler's object layout even when BOOST_FIBERS_AWAKENED_FROM_REMOTE is not defined, despite the comment indicating it's only needed for that feature. This adds size/ABI surface and can trigger -Wunused-private-field warnings in builds that don't enable the macro. Consider making the member conditional like remote_ready_splk_ so default builds keep the previous layout.
    // When BOOST_FIBERS_AWAKENED_FROM_REMOTE is defined, worker_splk_
    // serializes attach/detach_worker_context so a remote thread calling
    // ctx->detach() inside awakened_from_remote() doesn't race the owning
    // thread's attach of another fiber.
    detail::spinlock                                            worker_splk_{};

src/scheduler.cpp:171

  • The awakened_from_remote fast-path can be invoked for main-context wakeups too (schedule_from_remote explicitly allows signaling main-context). If an algorithm implements awakened_from_remote by detaching/migrating contexts, calling ctx->detach() on a main-context will hit detach_worker_context() and assert because main-contexts are not in worker_queue_. Consider restricting the fast-path to worker_context instances so main-context wakeups continue to use the existing remote_ready_queue_ path.
    if ( ! ctx->is_context( type::pinned_context) ) {

src/scheduler.cpp:170

  • New behavior is introduced under BOOST_FIBERS_AWAKENED_FROM_REMOTE (bypassing remote_ready_queue_ via algorithm::awakened_from_remote), but there are no existing tests referencing awakened_from_remote. Given the repo has multithreaded scheduling tests, it would be good to add a focused test that enables the macro and verifies (1) awakened_from_remote is called for remote wakeups, and (2) the fallback remote_ready_queue_ path is still used for pinned contexts / when awakened_from_remote returns false.
#if defined(BOOST_FIBERS_AWAKENED_FROM_REMOTE)
    // Let the algorithm handle cross-thread scheduling directly if it
    // supports a thread-safe ready-queue (e.g. a shared priority queue).
    // The subsequent ctx->detach() inside awakened_from_remote() is
    // serialized against the owning thread's attach by worker_splk_.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants