-
Notifications
You must be signed in to change notification settings - Fork 2
Context Parallel Serving: establish SGLang feasibility and target design #5
Copy link
Copy link
Open
Labels
area:servingContext Parallel Serving and additional serving integrations.Context Parallel Serving and additional serving integrations.researchEvidence gathering, compatibility investigation, or an exploratory prototype.Evidence gathering, compatibility investigation, or an exploratory prototype.roadmapA direction-level issue with an explicitly bounded initial stage.A direction-level issue with an explicitly bounded initial stage.
Description
Activity
Metadata
Metadata
Assignees
Labels
area:servingContext Parallel Serving and additional serving integrations.Context Parallel Serving and additional serving integrations.researchEvidence gathering, compatibility investigation, or an exploratory prototype.Evidence gathering, compatibility investigation, or an exploratory prototype.roadmapA direction-level issue with an explicitly bounded initial stage.A direction-level issue with an explicitly bounded initial stage.
Outcome and baseline
Support SGLang Prefill and Decode Context Parallelism with degree greater than
one while preserving FFN row ownership and Elastic KV Cache capacity semantics.
CrossPool currently rejects these modes, and its attention topology and FFN
handoff use TP/DP rank geometry. Removing validation alone would leave the
ownership and capacity contracts unresolved.
This issue covers the SGLang support audit, installed-serving prototype, and
target-design recommendation. Production integration is subsequent work.
Suggested implementation route
support. Treat Prefill and Decode Context Parallelism separately. Identify
runnable configurations, row partition/gather behavior, KV position
partitioning, and eager/captured execution requirements.
row ownership, KV Capacity Group membership, and graph behavior for the
proposed modes. Retain rank-local shape and protocol evidence rather than
inferring correctness from successful HTTP completion alone.
contracts. Specify exactly which model/mode/topology combinations proceed
and the numerical, graph, serving, and reclamation evidence each requires.
capacity changes at their owners and admit only the designed modes.
serving completion, and failure/shutdown for each selected combination.
Discovery completion evidence
model, mode, or contract that prevents proceeding.
Missing resources leave required experiments incomplete. Existing
model-qualification and installed-serving harnesses are reusable evidence
surfaces; existing TP support does not establish context-parallel support.
References