Skip to content

feat(task): implement GraphResolver with fan-in cycle pruning and migrate monorepo to Task System v3 - #976

Merged
kyasbal merged 35 commits into
GoogleCloudPlatform:epic/task-system-v3from
kyasbal:push-tloptkompyoq
Sep 10, 2026
Merged

feat(task): implement GraphResolver with fan-in cycle pruning and migrate monorepo to Task System v3#976
kyasbal merged 35 commits into
GoogleCloudPlatform:epic/task-system-v3from
kyasbal:push-tloptkompyoq

Conversation

@kyasbal

@kyasbal kyasbal commented Sep 9, 2026

Copy link
Copy Markdown
Member

Summary

This PR implements the new GraphResolver with priority-based fan-in cycle pruning and completes the monorepo migration to Task System v3 as part of epic/task-system-v3.

Key Changes

  • 4-Phase GraphResolver (pkg/core/task):
    • Phase 1 (Mandatory Dependency Closure): BFS anchor discovery expanding required point-to-point and fan-in dependencies.
    • Phase 2 (Fan-In Candidate Binding): Binds candidate fan-in dependencies by tag and dependency scope with WithTagPriority support.
    • Phase 3 (Optional Binding & Mandatory 1:1 Edges): Evaluates optional point-to-point dependencies against the active graph and builds mandatory 1:1 edges.
    • Phase 3.5 (Fan-In Cycle Resolution, Pruning & Edge Deduplication): Detects circular fan-in dependencies, enforces AllowMultiStageExecution label requirements, checks for ambiguous priority ties, and deterministically prunes lower-priority cyclic fan-in edges when higher-priority bootstrap producers exist.
    • Phase 4 (Topological Sorting): Executes Kahn's algorithm over the resolved DAG to produce a deterministic execution order in TaskSet.
  • Runner & Taskbase Modernization (pkg/core/inspection, pkg/core/task):
    • Updated LocalRunner to execute ResolveGraph, track concrete TaskEdge metadata, and manage memory lifecycle.
    • Modernized TaskBase, InventoryTask, and parser tasks to adopt the new Dependency model and edge attributes.
  • Monorepo Inspection Task Migration (pkg/task/inspection):
    • Migrated all inspection tasks, parsers, and subgraphs across GCP, GKE, Composer, and OSS cluster packages to the v3 task interfaces and builder primitives.

Testing

  • Exhaustive table-driven unit tests for GraphResolver and cycle resolution in graphresolver_test.go and graphresolver_cycle_test.go.
  • Test cases covering priority-based pruning (1-hop, self-loop, and multi-hop transitive cycles), fail-fast error conditions (ambiguous priority ties, missing multi-stage label, lack of bootstrap producer), edge attribute deduplication/upgrades, and 100-iteration random shuffle invariance.
  • All backend unit tests pass cleanly across the monorepo (make test-go).

@kyasbal kyasbal added type:feature New feature or request area:backend-core Core backend frameworks, DAG task runner, and server API labels Sep 9, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the task graph resolution mechanism by replacing the iterative GraphResolver with a deterministic 4.5-phase resolution algorithm (ResolveGraph). It introduces structured Dependency descriptors (such as point-to-point and fan-in tags with varying scopes) to replace taskid.UntypedTaskReference, adds robust cycle detection with priority-based pruning, and migrates the inventory system to a demand-driven tag-based approach. Additionally, the LocalRunner is optimized to clean up intermediate results before releasing waiter locks to prevent peak heap bloat. Feedback on the changes suggests standardizing on the coretask import alias instead of common_task in several form builder files to maintain code consistency.

Comment thread pkg/core/inspection/formtask/fileform.go Outdated
Comment thread pkg/core/inspection/formtask/setform.go Outdated
Comment thread pkg/core/inspection/formtask/textform.go Outdated
Comment thread pkg/core/inspection/taskbase/inventory_task.go
Comment thread pkg/core/inspection/taskbase/inventory_task_test.go
Comment thread pkg/core/task/graphresolver_cycle_test.go
@kyasbal
kyasbal marked this pull request as ready for review September 10, 2026 06:54
@kyasbal
kyasbal merged commit eee3b96 into GoogleCloudPlatform:epic/task-system-v3 Sep 10, 2026
10 of 11 checks passed
@kyasbal
kyasbal deleted the push-tloptkompyoq branch September 10, 2026 06:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:backend-core Core backend frameworks, DAG task runner, and server API type:feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant