Skip to content

Add TunnelledAddress and tagged address forms - #505

Open
goodboy wants to merge 8 commits into
ng_tpts_planningfrom
wkt/addr_unpacking
Open

Add TunnelledAddress and tagged address forms#505
goodboy wants to merge 8 commits into
ng_tpts_planningfrom
wkt/addr_unpacking

Conversation

@goodboy

@goodboy goodboy commented Aug 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Add TunnelledAddress and tagged addrs

Motivation

Tractor's address layer previously flattened WireGuard maddrs to
their overlay socket address and serialized every transport as an
untagged pair. The runtime could neither retain tunnel metadata until
network setup nor distinguish transport protocols from the serialized
shape.

This change keeps tunnel declarations as transparent wrappers until
the socket bind/connect boundaries and gives TCP and Unix addresses
explicit wire tags. Legacy tuple and msgpack-list inputs remain
readable during the migration.

Src of research

The implementation follows the shared backend contract and plan 03's
three-layer WireGuard model. Layer A uses py-multiaddr's merged
WireGuard codec and treats a tunnel as bindspace metadata around an
existing L4 address, rather than inventing another message transport.

Summary of changes

  • Add frozen WGTunnelSpec and TunnelledAddress structures with
    nested maddr parsing, formatting, namespace and bindspace
    delegation, and recursive strip_tunnels() support.
  • Preserve wrappers through declaration and discovery, then peel them
    immediately before Channel dialing and Endpoint listener
    binding.
  • Replace the example-specific WireGuard parser with the discovery
    API and pin py-multiaddr's merged WireGuard codec revision.
  • Model serializable bindspace declarations separately from scoped
    live capabilities, namespace identity, ownership and teardown
    resources.
  • Decode canonical ('tcp', host, port), ('unix', path) and
    input-only ('uds', path) forms while retaining legacy pair and
    native IPv6 readers.
  • Switch .unwrap() emission, spawn payloads, multiaddr composition
    and runtime comparisons to canonical tagged forms.
  • Add tunnel, boundary, tagged serialization, mixed transport and
    nested maddr regression coverage.
  • Verify 412 non-debugger TCP tests, 410 non-debugger UDS tests and
    45 debugger/devx tests. A downstream Piker runtime boot and tagged
    registry regression also pass against this head.

Scopes changed

  • tractor.discovery._tunnel owns tunnel declarations, nested
    parsing and transparent overlay delegation.
  • tractor.discovery._addr and ._multiaddr own tagged
    serialization compatibility and composed maddr conversion.
  • tractor.ipc._chan and ._server enforce the tunnel-peeling
    socket boundary without changing transport dispatch.
  • tractor.ipc._tcp, ._uds and tractor.msg.types.SpawnSpec emit
    and carry protocol-specific tagged tuple shapes.
  • examples.multihost.wg_lan consumes discovery's canonical
    WireGuard parser instead of maintaining a second grammar.

Future follow up

  • Land the downstream Piker compatibility patch. Normalize legacy
    and tagged registrar values in piker.service._registry and retain
    canonical runtime comparisons.
  • Implement WireGuard layer B in a separate tractor PR. Replace
    wg(8) subprocess reads with focused pyroute2 queries under Trio
    without introducing another event loop.
  • Implement WireGuard layer C in a separate tractor PR. Realize
    scoped bindspace capabilities, enter namespaces during child
    bootstrap, drop provisioning authority and test owned versus
    borrowed teardown under CAP_NET_ADMIN.
  • Replace the py-multiaddr Git revision with a release floor
    after the merged WireGuard codec ships and finish protocol
    registration through tractor issue wg multiaddr protocol: upstream spec submission plan #483.

Links

(this pr content was generated in some part by opencode using
gpt-5.6-sol (openai))

Copilot AI lite review requested due to automatic review settings August 20, 2026 16:18
This was referenced Aug 20, 2026
Open
Open
Open
Copilot AI reviewed Aug 20, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR advances Tractor’s addressing model by (1) adding native WireGuard (/wg/) multiaddr support via a TunnelledAddress wrapper that carries tunnel metadata locally while delegating wire serialization to the overlay transport address, and (2) migrating transport .unwrap() emission to canonical tagged tuples (e.g. ('tcp', host, port) and ('unix', path)) while preserving legacy input compatibility at decoding boundaries.

Changes:

  • Add tractor.discovery._tunnel with TunnelledAddress + WireGuard maddr parse/compose helpers, plus boundary utilities (strip_tunnels, tunnels_of).
  • Switch TCP/UDS address serialization to canonical tagged tuples and update discovery/multiaddr/IPC call sites + tests accordingly (retaining legacy pair inputs).
  • Ensure tunnel wrappers are peeled at IPC transport boundaries (server listen + channel connect), and update the multihost WG example and docs.

Reviewed changes

Copilot reviewed 39 out of 40 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
uv.lock Adds py-multibase to the lock to support WG key encoding/decoding.
tractor/msg/types.py Loosens spawn address tuple typing to accommodate protocol-specific tagged shapes validated at boundaries.
tractor/ipc/_uds.py Updates UDS unwrapped form to canonical ('unix', path) and broadens decoding to tagged/legacy inputs.
tractor/ipc/_tcp.py Updates TCP unwrapped form to canonical ('tcp', host, port) and adjusts callers that previously unpacked (host, port).
tractor/ipc/_server.py Broadens accepted listen address declarations and peels tunnel wrappers before Endpoint construction.
tractor/ipc/_chan.py Accepts tunnelled declarations and strips tunnels immediately before transport dispatch/connect.
tractor/discovery/_tunnel.py New module implementing WG tunnel spec, tunnelled address wrapper, WG maddr parsing/composition, and peeling helpers.
tractor/discovery/_multiaddr.py Adds WG-aware parsing/composition and updates TCP/UDS handling to the tagged unwrap contract.
tractor/discovery/_addr.py Introduces canonical tagged address aliases, updates wrapping/dispatch for tagged+legacy forms, and recognizes TunnelledAddress as wrapped.
tractor/discovery/init.py Re-exports tunnel and WG helpers from the discovery public API.
tractor/_root.py Strips tunnels when allocating random bind addrs derived from contacted registry addresses.
tests/test_local.py Updates assertions to compare canonical tagged unwrapped addresses.
tests/ipc/test_server_tunnel_boundary.py New regression ensuring tunnel wrappers are peeled before Endpoint reflection/dispatch.
tests/ipc/test_each_tpt.py Updates runtime registry address comparisons to canonical tagged serialization.
tests/ipc/test_channel_tunnel_boundary.py New regression ensuring tunnel wrappers are peeled before outbound transport lookup/connect.
tests/discovery/test_tunnelled_addr.py New tests for delegation semantics, peeling, tunnel enumeration, and serialization behavior.
tests/discovery/test_tpt_bind_addrs.py Updates TCP port assertions and disjointness checks for tagged unwrapped tuples.
tests/discovery/test_multiaddr.py Expands coverage for WG parsing/composition + updates TCP/UDS expectations to tagged unwrap shapes.
tests/discovery/test_address_serialization.py New focused tests for canonical tagged decoding and legacy compatibility.
pyproject.toml Pins unreleased WG codec revision for multiaddr in PEP 621 metadata and adds py-multibase; enables direct references for Hatch.
examples/multihost/wg_lan/wg_maddr.py Removes duplicate parser; keeps only explicit impure WG peer verification using production parser types.
examples/multihost/wg_lan/README.md Updates documentation to match production tunnel wrapper model and dependency contract.
examples/multihost/wg_lan/host_b_client.py Migrates demo client to TunnelledAddress + updated transport enabling via overlay proto.
examples/multihost/wg_lan/host_a_srv.py Migrates demo server to TunnelledAddress + mk_maddr printing; adjusts key/bearer access.
ai/tpt-backends/03_wg_tunnel_bindspace.md Updates plan documentation to reflect implemented layer-A behavior and refined layer-C capability model.
ai/tpt-backends/00_shared_backend_contract.md Clarifies bindspace semantics vs. process namespace lifecycle (plan-03).
ai/prompt-io/opencode/20260820T033108Z_ba07e09d_prompt_io.raw.md Records agent prompt I/O for the tagged-writer migration step.
ai/prompt-io/opencode/20260820T033108Z_ba07e09d_prompt_io.md Summarizes that prompt I/O session (writers).
ai/prompt-io/opencode/20260820T033107Z_ba07e09d_prompt_io.raw.md Records agent prompt I/O for tagged-reader compatibility work.
ai/prompt-io/opencode/20260820T033107Z_ba07e09d_prompt_io.md Summarizes that prompt I/O session (readers).
ai/prompt-io/opencode/20260820T021516Z_dfad66a0_prompt_io.raw.md Records prompt I/O for bindspace capability design doc updates.
ai/prompt-io/opencode/20260820T021516Z_dfad66a0_prompt_io.md Summarizes that prompt I/O session (docs).
ai/prompt-io/opencode/20260819T213145Z_f81fc5e5_prompt_io.raw.md Records prompt I/O for inbound tunnel boundary integration.
ai/prompt-io/opencode/20260819T213145Z_f81fc5e5_prompt_io.md Summarizes that prompt I/O session (inbound boundary).
ai/prompt-io/opencode/20260819T213144Z_f81fc5e5_prompt_io.raw.md Records prompt I/O for outbound tunnel boundary integration.
ai/prompt-io/opencode/20260819T213144Z_f81fc5e5_prompt_io.md Summarizes that prompt I/O session (outbound boundary).
ai/prompt-io/opencode/20260818T075031Z_dd02c7c0_prompt_io.raw.md Records prompt I/O for WG multiaddr integration batch.
ai/prompt-io/opencode/20260818T075031Z_dd02c7c0_prompt_io.md Summarizes that prompt I/O session (WG integration).
ai/prompt-io/opencode/20260818T021729Z_d9a6e2e9_prompt_io.raw.md Records prompt I/O for tunnelled-address handoff completion.
ai/prompt-io/opencode/20260818T021729Z_d9a6e2e9_prompt_io.md Summarizes that prompt I/O session (handoff completion).

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tractor/ipc/_server.py
Comment on lines 963 to 970
async def listen_on(
self,
*,
accept_addrs: list[tuple[str, int|str]]|None = None,
accept_addrs: list[
UnwrappedAddress|Address|TunnelledAddress
]|None = None,
stream_handler_nursery: Nursery|None = None,
) -> list[Endpoint]:
@goodboy goodboy changed the title Wkt/addr unpacking Add TunnelledAddress and tagged address forms Aug 20, 2026
@goodboy goodboy mentioned this pull request Aug 21, 2026
Open
35 tasks
@goodboy
goodboy force-pushed the wkt/addr_unpacking branch from 5d92595 to 7d0b84e Compare August 28, 2026 19:18
@goodboy
goodboy force-pushed the wkt/addr_unpacking branch from 7d0b84e to 5b606ba Compare August 30, 2026 02:01
goodboy added 8 commits August 29, 2026 22:03
@goodboy
Add TunnelledAddress wrapper primitives
fa8067f 
Introduce the first layer-A address type from the `wg` bindspace
plan without treating a transparent tunnel as a `MsgTransport`.

Deats,
- add frozen `WGTunnelSpec` and `TunnelledAddress` structs which
  delegate proto identity, bindspace, validity and wire
  serialization to their overlay
- add `strip_tunnels()` and `tunnels_of()` for nested wrappers
- recognize wrappers in `is_wrapped_addr()` while keeping them out
  of `_address_types`
- cover delegation, namespace fallback and nested peeling semantics

Also,
- widen `Address.namespace` ids for named netns
- export the new discovery API
- clarify that tractor's layer-C bindspace lifecycle may provision
  the kernel-owned bearer without making it a `MsgTransport`

Prompt-IO: ai/prompt-io/opencode/20260818T021729Z_d9a6e2e9_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
@goodboy
Support nested wg maddrs
c7451bb 
Teach discovery to preserve WireGuard bearer and identity metadata
around a bindable TCP overlay.

Deats,
- encode `wg(8)` keys as strict 32-byte multibase values
- peel nested stacks with `Multiaddr.decapsulate_code()` and compose
  them with `.encapsulate()` instead of splitting strings
- integrate wrappers with `parse_maddr()`, `mk_maddr()`,
  `wrap_address()` and `parse_endpoints()`
- pin the unreleased py-multiaddr#108 codec in package metadata
- cover exact round trips, nesting, bad grammar and missing codecs

Prompt-IO: ai/prompt-io/opencode/20260818T075031Z_dd02c7c0_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
@goodboy
Use discovery's wg parser in examples
bcf8f87 
Drop the example-local address struct and hand-rolled single-tunnel
parser now that discovery owns the production implementation.

Keep only the explicit `wg(8)` peer probe in the multihost helper,
and update the examples and plan for nested parsing, packaged codec
dependencies and tractor-owned bindspace provisioning.

Prompt-IO: ai/prompt-io/opencode/20260818T075031Z_dd02c7c0_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
@goodboy
Peel tunnels before Channel connects
6b8e9ad 
Retain tunnel annotations through address declaration, then hand only
the bindable overlay to exact-type transport lookup and dialing.

Broaden `Channel.from_addr()` and `_connect_chan()` inputs accordingly,
and cover plain plus tunnelled TCP dispatch arguments.

Prompt-IO: ai/prompt-io/opencode/20260819T213144Z_f81fc5e5_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
@goodboy
Peel tunnels before Endpoint binding
176e0c3 
Carry tunnel declarations through listener configuration, then strip
them immediately before constructing transport endpoints.

Also allocate random listener addresses from a contacted registry's
overlay, and prove a real TCP listener never stores the wrapper while
the source declaration retains its bindspace metadata.

Prompt-IO: ai/prompt-io/opencode/20260819T213145Z_f81fc5e5_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
@goodboy
Model bindspaces as scoped capabilities
419ce6a 
Separate serializable bindspace declarations from live namespace
identity, FDs, ownership and teardown resources.

Require child namespace entry during spawn bootstrap, before actor
runtime initialization, then distinguish listen/dial provisioning and
owned/borrowed cleanup without encoding operation role into maddrs.

Prompt-IO: ai/prompt-io/opencode/20260820T021516Z_dfad66a0_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
@goodboy
Decode tagged transport addresses
0a74128 
- Define canonical `tcp` and `unix` tuple shapes while retaining
  legacy pair aliases as the emitted `UnwrappedAddress`.

- Dispatch tagged tuple/list payloads explicitly, accept `uds` as a
  Unix input alias, and preserve legacy TCP, UDS, and native IPv6
  readers.

- Cover tag aliases, msgpack-style lists, legacy payloads, and IPv6
  socket addresses before switching writers.

Prompt-IO: ai/prompt-io/opencode/20260820T033107Z_ba07e09d_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
@goodboy
Emit canonical tagged addresses
51f4e31 
- Make `TCPAddress.unwrap()` emit `('tcp', host, port)` and
  `UDSAddress.unwrap()` emit `('unix', path)` while retaining the
  compatibility readers from the preceding change.

- Pass concrete TCP fields to Trio, compose multiaddrs from tagged
  values, and let `SpawnSpec` carry protocol-specific tuple shapes
  for validation by `wrap_address()`.

- Compare runtime, registry, bind, and tunnel addresses through
  canonical serialized forms and cover both TCP and UDS operation.

Prompt-IO: ai/prompt-io/opencode/20260820T033108Z_ba07e09d_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
@goodboy
goodboy force-pushed the wkt/addr_unpacking branch from 5b606ba to 51f4e31 Compare August 30, 2026 02:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@goodboy