Store cross-version processes in an external context dict

[GlassOfWhiskey]

Instead of plugging cross-version CWL step.run processes directly into parent WorkflowStep objects (which is not supported), keep them in an external context: dict[str, tuple[Process, bool]] keyed by step URI. Steps with non-inline processes keep a string run reference resolved through the context.

Add get_step_uri and an AbstractProcess type alias, and update load_step, empty_inputs, type_for_source, and the per-version refactor modules to consult the shared context so they no longer need to load or embed cross-version processes inline.

This changes the API of expression_refactor.refactor: it now writes one output file per step whose process cannot be directly embedded into the parent WorkflowStep object.

[codecov]

Codecov Report

❌ Patch coverage is 45.59387% with 142 lines in your changes missing coverage. Please review.
✅ Project coverage is 38.67%. Comparing base (4c8058e) to head (ccf31ba).

Files with missing lines	Patch %	Lines
src/cwl_utils/cwl_v1_1_expression_refactor.py	21.27%	36 Missing and 1 partial ⚠️
src/cwl_utils/cwl_v1_0_expression_refactor.py	23.40%	35 Missing and 1 partial ⚠️
src/cwl_utils/cwl_v1_2_expression_refactor.py	37.25%	27 Missing and 5 partials ⚠️
src/cwl_utils/expression_refactor.py	71.62%	9 Missing and 12 partials ⚠️
src/cwl_utils/parser/utils.py	57.14%	3 Missing and 3 partials ⚠️
src/cwl_utils/parser/cwl_v1_1_utils.py	50.00%	0 Missing and 3 partials ⚠️
src/cwl_utils/parser/cwl_v1_2_utils.py	50.00%	0 Missing and 3 partials ⚠️
src/cwl_utils/parser/cwl_v1_0_utils.py	66.66%	0 Missing and 2 partials ⚠️
src/cwl_utils/utils.py	66.66%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #459      +/-   ##
==========================================
- Coverage   38.78%   38.67%   -0.12%     
==========================================
  Files          50       50              
  Lines       34618    34653      +35     
  Branches     8802     8813      +11     
==========================================
- Hits        13428    13402      -26     
- Misses      18306    18388      +82     
+ Partials     2884     2863      -21     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[mr-c]

$ cwl-expression-refactor t src/cwl_utils/testdata/wc3-tool_v1_0.cwl
Processing src/cwl_utils/testdata/wc3-tool_v1_0.cwl.
Traceback (most recent call last):
  File "/home/michael/cwl-utils/env3.13/bin/cwl-expression-refactor", line 6, in <module>
    sys.exit(main())
             ~~~~^^
  File "/home/michael/cwl-utils/src/cwl_utils/expression_refactor.py", line 683, in main
    sys.exit(run(sys.argv[1:]))
             ~~~^^^^^^^^^^^^^^
  File "/home/michael/cwl-utils/src/cwl_utils/expression_refactor.py", line 688, in run
    return refactor(parse_args(args))
  File "/home/michael/cwl-utils/src/cwl_utils/expression_refactor.py", line 705, in refactor
    result, modified = cwl_v1_0_expression_refactor.traverse(
                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        cwl_v1_0.load_document_by_yaml(result, uri),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
        context,
        ^^^^^^^^
    )
    ^
  File "/home/michael/cwl-utils/src/cwl_utils/cwl_v1_0_expression_refactor.py", line 304, in traverse
    result, modified = traverse_workflow(
                       ~~~~~~~~~~~~~~~~~^
        workflow, replace_etool, skip_command_line1, skip_command_line2, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/michael/cwl-utils/src/cwl_utils/cwl_v1_0_expression_refactor.py", line 2043, in traverse_workflow
    step_modified = traverse_step(
        step,
    ...<9 lines>...
        context,
    )
  File "/home/michael/cwl-utils/src/cwl_utils/cwl_v1_0_expression_refactor.py", line 1874, in traverse_step
    clt_modified = traverse_CommandLineTool(
        original_process,
    ...<6 lines>...
        context,
    )
  File "/home/michael/cwl-utils/src/cwl_utils/cwl_v1_0_expression_refactor.py", line 1448, in traverse_CommandLineTool
    context[get_step_uri(new_clt_step)] = (new_clt, True)
            ~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/home/michael/cwl-utils/src/cwl_utils/utils.py", line 459, in get_step_uri
    raise Exception(
        f"Impossible to retrieve URI for step {step.id}: it embeds a process"
    )
Exception: Impossible to retrieve URI for step main: it embeds a process

[mr-c]

@GlassOfWhiskey This is better, while cwl-expression-refactor t src/cwl_utils/testdata/wc3-tool_v1_0.cwl no longer fails, the output is not real CWL

Cherry-picking the commits from #458 helps, but here is the current result:

#!/usr/bin/env cwl-runner
id: file:///home/michael/cwl-utils/src/cwl_utils/testdata/wc3-tool_v1_0.cwl
class: Workflow
inputs:
- id: file1
  type:
    items: ''
    type: array
outputs:
- id: output
  outputSource: ''
  type: int
requirements:
- class: SubworkflowFeatureRequirement
cwlVersion: v1.0
steps:
- id: '#main'
  in:
  - id: ''
    source: file1
  out:
  - ''
  run:
    id: ''
    class: ''
    inputs:
    - id: ''
      type:
        items: ''
        type: array
    outputs:
    - id: ''
      outputSource: ''
      type: int
    cwlVersion: ''
    steps:
    - id: ''
      in:
      - id: ''
        source: file1
      out:
      - ''
      run:
        id: ''
        class: ''
        inputs:
        - id: ''
          inputBinding: {}
          type:
            items: ''
            type: array
        outputs:
        - id: ''
          outputBinding:
            glob: output.txt
          type:
            items: ''
            type: array
        hints:
        - class: ''
          ramMin: 8
        cwlVersion: ''
        stdout: output.txt
    - id: ''
      in:
      - id: ''
        source: file1
      - id: ''
        source: ''
      out:
      - id: ''
      run:
        id: ''
        class: ''
        inputs:
        - id: ''
          type:
            items: ''
            type: array
            inputBinding:
              loadContents: true
        - id: ''
          type:
            items: ''
            type: array
        outputs:
        - id: ''
          type: int
        requirements:
        - class: ''
        cwlVersion: ''
        expression: |-
          ${
            var self=inputs.self;
            return {"result": function(){
            var s = self[0].contents.split(/\r?\n/);
            return parseInt(s[s.length-2]);
          }()};
           }