Skip to content

Fix CIDConflict - #44

Merged
mzueva merged 2 commits into
mainfrom
mzueva/fix-cid-conflict
May 25, 2026
Merged

Fix CIDConflict#44
mzueva merged 2 commits into
mainfrom
mzueva/fix-cid-conflict

Conversation

@mzueva

@mzueva mzueva commented May 25, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes unstable Content IDs (CIDs) caused by non-deterministic Tengo map iteration in the import workflow templates, and migrates model/ and ui/ from the legacy vite + vue-tsc build pipeline to @milaboratories/ts-builder.

  • CID fix: All import templates replace the unordered for key, file in map / for key, value in map idiom with for key in maps.getKeys(map) to guarantee sorted, deterministic iteration — affecting input-file frame registration and column-rename expression ordering. import-immunoSeq.tpl.tengo is partially fixed (column mapping only); the input-file loop still uses maps.forEach, which may reproduce the instability for that format.
  • Build migration: model/ and ui/ adopt ts-builder scripts (build, watch, type-check), standardised tsconfig inheritance via @milaboratories/ts-configs, and an explicit exports field in model/package.json; dependency versions across the whole workspace are bumped to the latest SDK releases.

Confidence Score: 3/5

The build migration and most workflow fixes are clean, but import-immunoSeq.tpl.tengo input-file iteration was not migrated to the sorted pattern, leaving immunoSeq imports potentially still vulnerable to the CID instability this PR targets.

The maps.forEach left in import-immunoSeq.tpl.tengo for input-file ordering means the stated goal of the PR is only partially achieved for that format — users who re-import immunoSeq data could still see CID conflicts. The self-alias guard omission in import-custom.tpl.tengo is a narrower risk but could produce a runtime error if a user creates a passthrough mapping.

workflow/src/import-immunoSeq.tpl.tengo (input-file loop not migrated to sorted iteration), workflow/src/import-custom.tpl.tengo (missing self-alias guard)

Important Files Changed

Filename Overview
workflow/src/import-common.tpl.tengo Both input-file and column-mapping loops switched to maps.getKeys sorted iteration; self-alias guard added to renameExpressions. Core fix is correct and complete.
workflow/src/import-immunoSeq.tpl.tengo Column-mapping loop fixed to use maps.getKeys, but input-file iteration still uses maps.forEach — diverges from the sorted-iteration pattern applied everywhere else and may still produce unstable CIDs for immunoSeq imports.
workflow/src/import-qiagen.tpl.tengo Both input-file and column-mapping loops migrated to sorted maps.getKeys iteration. Change is consistent with the fix pattern.
workflow/src/import-custom.tpl.tengo Both loops converted to maps.getKeys; maps import added. Missing the self-alias guard that import-common received, which could fail if a user maps a column to its own name.
workflow/src/infer-columns-custom.lib.tengo Two map-iteration loops (customMapping and columnMapping) migrated to maps.getKeys for deterministic ordering; minor comment/whitespace cleanup.
model/package.json Build pipeline migrated from vite + vite-plugin-dts to @milaboratories/ts-builder; exports field added; dependencies updated to match newer SDK versions.
ui/package.json Build pipeline migrated to ts-builder; vue-tsc, vite, @vitejs/plugin-vue removed; @milaboratories/helpers and ts-configs added.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["inputFiles.inputs()"] --> B{Template}
    B -->|"import-common / import-qiagen / import-custom"| C["maps.getKeys sorted iteration ✅"]
    B -->|"import-immunoSeq"| D["maps.forEach non-deterministic ⚠️"]
    C --> E["Frame IDs stable"]
    D --> F["Frame IDs potentially unstable"]
    E --> G["maps.getKeys on columnMapping ✅"]
    F --> G
    G --> H["withColumns / filter / aggregate"]
    H --> I["save chain.tsv"]
    E --> J["Stable CID ✅"]
    F -.-> K["CID conflict risk ⚠️"]
Loading

Comments Outside Diff (1)

  1. workflow/src/import-immunoSeq.tpl.tengo, line 27-34 (link)

    P1 Input file iteration not migrated to sorted maps.getKeys

    Every other import template in this PR (import-common, import-qiagen, import-custom) switched from for key, file in inputFiles.inputs() to for key in maps.getKeys(inputMap) to guarantee deterministic ordering. import-immunoSeq.tpl.tengo still uses maps.forEach, which iterates in hash/insertion order and produces non-deterministic frame IDs, leaving immunoSeq imports exposed to the same unstable-CID problem this PR sets out to fix.

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: workflow/src/import-immunoSeq.tpl.tengo
    Line: 27-34
    
    Comment:
    **Input file iteration not migrated to sorted `maps.getKeys`**
    
    Every other import template in this PR (`import-common`, `import-qiagen`, `import-custom`) switched from `for key, file in inputFiles.inputs()` to `for key in maps.getKeys(inputMap)` to guarantee deterministic ordering. `import-immunoSeq.tpl.tengo` still uses `maps.forEach`, which iterates in hash/insertion order and produces non-deterministic frame IDs, leaving immunoSeq imports exposed to the same unstable-CID problem this PR sets out to fix.
    
    How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
workflow/src/import-immunoSeq.tpl.tengo:27-34
**Input file iteration not migrated to sorted `maps.getKeys`**

Every other import template in this PR (`import-common`, `import-qiagen`, `import-custom`) switched from `for key, file in inputFiles.inputs()` to `for key in maps.getKeys(inputMap)` to guarantee deterministic ordering. `import-immunoSeq.tpl.tengo` still uses `maps.forEach`, which iterates in hash/insertion order and produces non-deterministic frame IDs, leaving immunoSeq imports exposed to the same unstable-CID problem this PR sets out to fix.

### Issue 2 of 2
workflow/src/import-custom.tpl.tengo:38-44
**Missing self-alias guard on column rename**

`import-common.tpl.tengo` received an explicit check (`if canonical != original`) to skip self-aliasing, because passing `col("x").alias("x")` to `withColumns` creates a duplicate column and causes a runtime error. `import-custom.tpl.tengo` uses the same loop pattern but lacks the guard. If a user's custom mapping maps a canonical name to the same source column, the workflow will fail.

```suggestion
    // Rename original columns to canonical IDs
    renameExpressions := []
    for canonical in maps.getKeys(columnsInfo.columnMapping) {
        original := columnsInfo.columnMapping[canonical]
        if original != undefined && canonical != original {
            renameExpressions = append(renameExpressions, pt.col(original).alias(canonical))
        }
    }
    if len(renameExpressions) > 0 {
        df = df.withColumns(renameExpressions...)
    }
```

Reviews (1): Last reviewed commit: "Fix CIDConflict" | Re-trigger Greptile

Greptile also left 1 inline comment on this PR.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request addresses unstable CIDs by implementing sorted map iteration across several Tengo templates to ensure deterministic output. Additionally, it migrates the model and ui packages from a legacy Vite/vue-tsc build pipeline to @milaboratories/ts-builder and updates various SDK dependencies. Feedback suggests adding explanatory comments to the new iteration logic in import-custom.tpl.tengo and import-qiagen.tpl.tengo to maintain consistency with documentation provided in other workflow files.

Comment thread workflow/src/import-custom.tpl.tengo
Comment thread workflow/src/import-qiagen.tpl.tengo
Comment thread workflow/src/import-custom.tpl.tengo Outdated
@mzueva
mzueva added this pull request to the merge queue May 25, 2026
Merged via the queue into main with commit 99a0b47 May 25, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant