Skip to content

fix(swarm): qualify waiting-request outbound stream ownership - #1

Draft
caniko wants to merge 57 commits into
masterfrom
fix/cannot-extract-twice
Draft

caniko wants to merge 57 commits into
masterfrom
fix/cannot-extract-twice

Conversation

@caniko

@caniko caniko commented Oct 4, 2026 •

Copy link
Copy Markdown
Owner

Hosted CI carrier for libp2p/rust-libp2p#6427 at 0c6b55925053f4c5a859448a5828e076f9efabaa. The connection selects a waiting request before polling the outbound muxer, so completed or absent requests cannot cause an unowned stream to be opened and discarded. Regressions cover request-free polling and two requested streams with an additional stream available.

Upstream Continuous integration and Interoperability Testing runs require maintainer approval. Fork Actions are enabled, but GitHub currently lists zero discovered workflows and dispatch returns 404; this carrier has no executed application qualification. The remaining upstream ownership thread stays open. No local tests or evaluations were run during this maintenance pass.

caniko and others added 30 commits May 7, 2026 08:46
`Connection::poll` selects the next outbound-ready `SubstreamRequested`
via `iter_mut().next()` and calls `.extract()` on it. `extract()` moves
the data out and replaces the variant with `Done`, then relies on
`extracted_waker.wake()` to schedule the cleanup poll that removes the
entry from `requested_substreams`.

The waker is only populated by the `Future::poll` impl when the entry
first observes a `Poll::Pending`. If the entry is extracted before that
first poll ever happens — which can occur when `FuturesUnordered`'s
ordering puts a freshly-pushed entry at the head of `iter_mut()`, or
when the muxer returns `poll_outbound = Ready` on the same loop
iteration as the request was pushed — the waker is `None`, no wake is
scheduled, and the `Done` entry persists.

The next outbound-ready iteration's `iter_mut().next()` lands on that
stale `Done` entry and `extract()` panics.

The intent at this site has always been "find a request waiting for a
substream"; making that explicit with `iter_mut().find(|r| matches!(r,
SubstreamRequested::Waiting { .. }))` removes the dependence on
`FuturesUnordered`'s scheduling. Eager removal of the `Done` entry is
not possible through `FuturesUnordered`'s public API while iterating
it, so the existing wake-driven cleanup stays in place. The number of
un-cleaned `Done` entries is bounded by outstanding outbound requests
and is drained naturally as later events trigger connection polls.

Adds a unit test reproducing the failure shape directly on a
`FuturesUnordered<SubstreamRequested<()>, DeniedUpgrade>>`.
When the local node has an active reservation with a relay, it notifies swarm of the new external address. When calling `Swarm::remove_listener` on a listener with an active relay reservation, the address would still persist as an external address despite the local node not considered listening on the relay. This PR changes it so the when the node stops listening on the relay, it removes the external address if there is no other reservation to replace it.

resolves libp2p#6165.

Pull-Request: libp2p#6285.
This PR improves the `semver-checks` CI job by introducing caching to speed up executions and removing the explicit `cargo semver-checks` invocation, as it is already handled by the action by default.

The initial run with a cold cache is expected to take approximately 23 minutes, which is unchanged from the current execution time as the cache is being populated,  subsequent runs with a warm cache are expected to complete in approximately 3–6 minutes which reduces the job execution time by approximately 74–87% for cached runs.

Pull-Request: libp2p#6437.
- Extract `futures-timer` and `rand` as workspace dependencies
- Update all dependencies where the version bump was trivial.

Pull-Request: libp2p#6355.
Implement more stuff for `MessageAcceptance`: Clone, Copy, PartialEq, Eq, Hash

Pull-Request: libp2p#6445.
…bp2p#6439)

## Description
When the `partial_messages` feature is enabled, `Event::Subscribed` now
carries `supports_partial` and `requests_partial` flags from the remote
peer's `SubscriptionOpts`, so applications can react to a peer's partial
message capabilities at subscription time.
The internal state no longer tracks sent partial messages only for subscribed topics, but for all topics for which the user enabled partial messages.

This fixes issues such as
- sending both full and partial messages to a peer when publishing to a non-subscribed topic
- forwarding partials to the application and creating local state when a partial is received for an unsupported topic.

Pull-Request: libp2p#6422.
To avoid downstream dependency issues, bump wasm-bindgen-futures as high as possible.

Pull-Request: libp2p#6450.
The reason tests were failing was  ordering in the SDP offer/answer negotiation sequence.
We were creating the negotiated noise data channel after offer/answer setup.

With 0.17, that timing is no longer safe because SCTP startup depends on SDP state that needs the datachannel/application section to be present in time.
`create_data_channel` and `set_remote_description` communicate through shared `RTCPeerConnection` state, and  `RTCPeerConnection` committed local SDP before `create_data_channel` updated its internal datachannel/SCTP state.

This was fixed by moving negotiated noise data-channel creation to happen before SDP offer/answer generation, so SCTP startup in `webrtc` `0.17` sees valid sctp-port state and the channel reliably opens.

Superseeds libp2p#6391.
cc @ackintosh

Pull-Request: libp2p#6429.
libp2p#6409 missed the id truncation that existed before, this PR re-adds it

Pull-Request: libp2p#6428.
## Description

In [libp2p#3312](libp2p#3312), we migrated
from `prost` to `quick-protobuf` primarily to eliminate the need for the
`protoc` dependency on CI for generating Rust code from `.proto` files,
as we were not versioning them at the time.

However, `quick-protobuf` has been [unmaintained for
years](tafia/quick-protobuf#263), and we
should transition to a more actively maintained protobuf implementation.
By versioning the generated Rust code from the `.proto` files, we can
limit the need for `protoc` to only when the `.proto` files are modified
and the Rust code needs to be regenerated.

To address this, @macladson has generously taken the time to draft a
migration plan to move us back to `prost`.

---------

Co-authored-by: Mac L <mjladson@pm.me>
Application-side race conditions might cause calls to `publish_partial` to arrive in an unintended order. Currently, gossipsub silently accepts partial and overwrites the previously cached value - causing stale data to be replied on incoming requests.

This PR uses `Metadata::update` to compare the cached with the newly published value, and only proceeds with the publish if there is any fresh data.

Pull-Request: libp2p#6458.
jxs and others added 25 commits June 3, 2026 14:36
closes libp2p#6377

This PR splits the `mappings` field into `mappings`, `add_requests`, and `remove_requests` to manage port mappings and in-flight request tracking in separate data structures.

The `network_interface_change` and `listener_with_multiple_addresses` tests added in this PR reproduce the issue described in libp2p#6377, and both now pass after the fix.

Pull-Request: libp2p#6459.
`futures_bounded::Delay::tokio` compiles on `wasm32-unknown-unknown` (tokio's `time` feature builds there) but panics at runtime: in the browser, futures are driven by `wasm-bindgen-futures` rather than a tokio runtime, so there is no tokio timer driver and `tokio::time::sleep` panics the first time it is polled. tokio's timer fundamentally cannot run on that target.

Switch the workspace `futures-bounded` dependency to the `futures-timer` feature and replace every `Delay::tokio(..)` with `Delay::futures_timer(..)` across autonat, dcutr, identify, kad, perf, relay and request-response. `futures-timer` works on both native and wasm — it already backs the project's wasm timers via `gloo-timers` (see the `wasm-bindgen-futures` pin in the root `Cargo.toml`).

Pull-Request: libp2p#6488.
Forwarded messages were not accounted in the `topic_msg_sent_*` metrics.

Furthermore, after this change, all code location matching on `RpcOut` handle `Publish` and `Forward` the same, so I unified them into `Publish`.

Pull-Request: libp2p#6502.
If poll_outbound_unpin returns Ready(substream) but all
requested_substreams entries are Done, do not unconditionally
continue — the substream had no matching request and the
loop would busy-spin. Fall through instead.
Published through GitHub’s verified commit API during the hosted-only PR maintenance pass.
@caniko

caniko commented Oct 4, 2026

Copy link
Copy Markdown
Owner Author

@greptileai review the current candidate at 0c6b559. This is the hosted CI carrier for upstream libp2p#6427. The account credit-limit prerequisite is tracked by the PR maintenance pass; hosted compilation and swarm regression evidence are still required.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

caniko has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@caniko

caniko commented Oct 7, 2026

Copy link
Copy Markdown
Owner Author

@codex review

Please review the complete carrier candidate at head 0c6b55925053f4c5a859448a5828e076f9efabaa against base 197663d7cd23c121db30b4cbe2557eb28e454142. The head is exactly the upstream libp2p#6427 head; the carrier base is older, so its full comparison also includes upstream drift. Review must not be represented as executed swarm or interoperability qualification.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-07T19:40:25.367278Z 0c6b559 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0c6b559250

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

PUBLIC_TCP_DELAY,
PUBLIC_QUIC_DELAY,
PUBLIC_OTHER_DELAY,
Duration::ZERO,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Offset public dials after the private group

When smart dialing receives both private and public addresses, both group_delays calls use a zero offset. SmartDial places every returned future into the same FuturesUnordered, so the first public and private attempts are polled and started together; their ordering in the vector provides no head start. This defeats the documented private-before-public ranking and can let a WAN path win or create an unnecessary parallel connection attempt, so the public group needs an offset derived from the private group's delays.

Useful? React with 👍 / 👎.

Protocol::Dns(dns) | Protocol::Dns4(dns) | Protocol::Dns6(dns) => Some(dns),
_ => None,
}) {
return dns == "localhost" || dns.ends_with(".localhost");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep non-local DNS addresses out of the private bucket

For a DNS-only address such as /dns/example.com/tcp/443, this returns false, so the !is_global_addr check places it in the private bucket; conversely, localhost returns true and, because it has no IP component, falls into the other bucket. With smart dialing this reverses the documented classification, giving public DNS addresses aggressive private-address timing while delaying local DNS addresses. The predicate should identify non-local DNS names as global/other and localhost names as non-global/private.

Useful? React with 👍 / 👎.

Comment on lines +301 to +302
let message_length = buf.len();
if message_length > max_message_size {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Validate only the current gossipsub frame length

When the read buffer contains more than one framed RPC, buf.len() measures all buffered frames rather than the first frame being validated. Two individually valid RPCs whose combined buffered size exceeds global_max_transmit_size are therefore rejected, and even a single payload exactly at the configured limit is rejected because its varint prefix is included despite the codec documenting that the limit excludes it. Decode the length prefix first and compare its declared payload length to the limit.

Useful? React with 👍 / 👎.

Comment on lines +307 to +308
if let Some((mapping, _state)) =
self.mappings.remove(&(protocol, addr.port()))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Cancel pending UPnP additions when a listener expires

If an address expires while its AddMapping is awaiting a gateway response or waiting for retry, it is absent from mappings, so this branch does nothing and leaves the matching add_requests entry alive. A retry can then map the closed listener, or a late Mapped response can install it and emit ExternalAddrConfirmed for an address that is no longer listening. The available-gateway path must cancel pending additions and arrange for any late successful mapping to be removed.

Useful? React with 👍 / 👎.

Comment on lines +191 to +192
if let Some((addr, ReservationStatus::Confirmed)) =
self.reservation_addresses.remove(&connection_id)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Handle late reservation responses after listener closure

If the final relay listener closes while its initial reservation or a renewal is still in flight, remove deletes the Pending reservation entry even though this pattern only emits an expiration for Confirmed. The handler request remains active, so a subsequent ReservationReqAccepted reaches get_mut(&connection).expect("Relay connection exist") and panics the swarm. Listener closure must either cancel the pending handler operation or retain state that safely ignores its late response.

Useful? React with 👍 / 👎.

Comment on lines +229 to 234
if self.mappings.contains_key(&(protocol, addr.port())) {
tracing::debug!(
multiaddress=%multiaddr,
mapped_multiaddress=%mapping.multiaddr,
"port from multiaddress is already mapped on the gateway"
);
return;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Track listeners that share an already mapped UPnP port

When listener A owns an active mapping and listener B reports another address with the same protocol and port, this early return discards B without recording that it does not own the mapping. If B later emits ExpiredListenAddr, the available-gateway path removes the entry solely by (protocol, port) and tears down A's still-valid mapping. Preserve listener ownership or verify the stored mapping's listener/address before processing expiration.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants