Skip to content

MVP: flag unresolved external dependencies during migration analysis #41

Description

@AlexDeMichieli

Why

During the Siemens Building X Core migration, the migrator translated visible CI syntax well, but some pipelines could still produce syntactically valid yet non-functional GitHub Actions workflows.

The blockers were not unique to GitLab:

  • Missing or deeply nested templates and inheritance: external templates, shared libraries, or inherited configuration can define jobs, variables, conditions, artifacts, and deployment behavior that are not visible in the analyzed files. Real pipelines expanded from a few hundred lines to several thousand once nested templates were resolved, so depth itself is a signal.
  • Opaque execution logic: jobs may delegate important behavior to custom container images, plugins, tasks, or binaries whose internals are unavailable.
  • Build-time or runtime external fetching: images or jobs may clone or download additional repositories, binaries, or scripts while running, adding dependencies that never appear in the CI file.
  • Source-platform coupling: referenced logic may depend on predefined variables, credentials, commands, or REST/GraphQL APIs from the source CI platform.
  • Absent scripts: pipelines may invoke repository scripts, Make targets, or other files that were not provided to the agent.
  • Technical debt and obsolete steps: pipelines accumulate deprecated steps, commented-out blocks, redundant stages, and unused secrets. Automation cannot tell which steps are still required, so it may faithfully carry forward logic that should be dropped.
  • Validation limits: syntax validation cannot prove that unseen runtime behavior was preserved.

The tool does not need to recursively resolve, fix, or prune these dependencies. It should detect likely boundaries, raise clear concerns, and avoid overstating completeness.

Proposed MVP

Add a platform-agnostic external dependency preflight to the shared Analyze phase. Using only provided/local files, the agent looks for:

  • referenced templates, includes, shared libraries, or inherited configuration whose targets are unavailable (or are nested unusually deep)
  • custom images, plugins, tasks, or binaries whose behavior is not visible
  • build-time or runtime fetching of external repositories, binaries, or scripts
  • referenced scripts, build files, or commands whose source is absent
  • visible source-platform variables, credentials, commands, or API calls that require adaptation for GitHub Actions
  • steps that appear deprecated, commented-out, or redundant, and secrets that may no longer be needed

Platform-specific skills may provide detection cues-for example GitLab include/extends, Jenkins shared libraries, Azure DevOps templates/tasks, CircleCI orbs, or Bitbucket pipes-but the behavior and report format remain in the shared migration core.

For each concern, add one row to the migration report:

Reference Concern Impact Action needed
Shared pipeline template Source not provided / deeply nested High Provide or review the template manually
Custom build image/plugin Internal behavior is opaque High Confirm required commands, variables, and credentials
Deployment script Uses source-platform integration High Adapt the integration before cutover
Legacy stage Possibly deprecated/unused Low Confirm with owners whether it is still needed

Then:

  1. Convert only behavior that is visible.
  2. Add a greppable TODO(migration) comment at each affected workflow step instead of inventing missing behavior.
  3. Ask for a missing local file when it is required to understand the pipeline.
  4. For remote, cross-repository, opaque, or possibly-obsolete dependencies, proceed with documented caveats and defer the judgement to a human.
  5. Report Migration complete with documented gaps when an execution-critical concern remains; use Migration complete only when no concerns remain.
  6. Expand only templates/configuration that were provided and are visible; report absent content rather than inferring it.

Minimal implementation

Keep the first iteration instruction-driven:

  • define the generic preflight behavior and result wording in the shared migration workflow/guardrails and CLI migration-core skill
  • add a small generic Dependency Preflight section to migration reports
  • add short source-platform cue lists to existing platform mapping skills only where needed

No parser, manifest, dependency graph, resolver, API integration, or hook enforcement is required for this MVP.

Trust boundaries

  • Never execute analyzed scripts or binaries.
  • Never pull or run referenced container images.
  • Never fetch remote or cross-repository content.
  • Record secret/variable names only, never values.
  • Never invent behavior for unavailable content.

Acceptance criteria

  • Shared Analyze guidance detects unavailable external dependencies for every supported source CI platform.
  • Platform skills provide cues without implementing different preflight behavior or output formats.
  • Reports list each concern, its likely impact, and a clear manual action.
  • Affected workflow steps contain a TODO(migration) marker instead of fabricated behavior.
  • Missing required local files cause the agent to ask for those files.
  • Remote/cross-repository/opaque concerns are reported without fetching or executing them.
  • Possibly deprecated or redundant steps are flagged for human review, not removed automatically.
  • Execution-critical concerns produce Migration complete with documented gaps, not an unqualified completion claim.
  • Guidance distinguishes syntax validation from behavioral completeness.

Non-goals

  • Recursively resolving or migrating subdependencies
  • Building a dependency graph or resolver engine
  • Fetching remote content or inspecting container filesystems
  • Executing source scripts, images, plugins, or tasks
  • Deciding whether steps or secrets are still needed, or automatically pruning obsolete steps (flag for human review only)
  • Guaranteeing behavioral equivalence through static analysis
  • Delivering a full preflight framework in the first iteration

Metadata

Metadata

Assignees

No one assigned

    Labels

    agentsNew and udpated AgentsautomationNew and udpated GitHub Actions workflows and other automation scriptsknowledgebaseNew and updated knowledge articles

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions