Bug when using `capycli bom map -o outmap` (v2.11.1)

[rfuentess]

I have been experiencing this bug since v2.10.0. I am testing basically the
following:

capycli bom map -cf ComponentCache.json  -rc --similar  -oa \
    --matchmode all-versions  -vv                           \
    -i rust_filtered.bom.cdx.json                           \
    -mr rust_filtered_mapped_sbom4rust.mapresult.sbom.json  \
    -o rust_filtered.mapped.bom.cdx.json

The bug occurs when CaPyCLI processes the -o rust_filtered.mapped.bom.cdx.json
output file directly:

Writing updated SBOM to rust_filtered.mapped.bom.cdx.json
Traceback (most recent call last):
  File "/home/rfuentess/env/yassi/bin/capycli", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/rfuentess/env/yassi/lib/python3.11/site-packages/capycli/main/cli.py", line 28, in main
    app.run(argv)
  File "/home/rfuentess/env/yassi/lib/python3.11/site-packages/capycli/main/application.py", line 159, in run
    self._run(argv)
  File "/home/rfuentess/env/yassi/lib/python3.11/site-packages/capycli/main/application.py", line 140, in _run
    handle_bom.run_bom_command(self.options)
  File "/home/rfuentess/env/yassi/lib/python3.11/site-packages/capycli/bom/handle_bom.py", line 89, in run_bom_command
    app5.run(args)
  File "/home/rfuentess/env/yassi/lib/python3.11/site-packages/capycli/bom/map_bom.py", line 989, in run
    new_bom = self.create_updated_bom(sbom, result)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rfuentess/env/yassi/lib/python3.11/site-packages/capycli/bom/map_bom.py", line 695, in create_updated_bom
    newitem = self.update_bom_item(None, match_item)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rfuentess/env/yassi/lib/python3.11/site-packages/capycli/bom/map_bom.py", line 531, in update_bom_item
    purl=PackageURL.from_string(purl),
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rfuentess/env/yassi/lib/python3.11/site-packages/packageurl/__init__.py", line 485, in from_string
    raise ValueError(f'purl is missing the required "pkg" scheme component: {repr(purl)}.')
ValueError: purl is missing the required "pkg" scheme component: '["pkg:cargo/clap_builder@4.5.60","pkg:cargo/clap@4.5.60"]'.

Based at that output, the issue is that at some point the map result
contains a list of strings instead of a single match. However, I cannot
provide additional details about the component triggering the bug, as the
purl entry, AFAIK, appears when CaPyCLI performs the mapping against SW360 and
does not match a single entry at the component lists from my Rust manifest.

Relevant code::

capycli/capycli/bom/map_bom.py

Lines 526 to 532 in 14ac5a3

	# create a new one
	if purl:
	component = Component(
	name=match.get("Name", ""),
	version=match.get("Version", ""),
	purl=PackageURL.from_string(purl),
	bom_ref=match.get("RepositoryId", ""))

[rfuentess]

And I understand now the reason of the bug. This is the output specific to my issue:

[
    {
        "Id": "cf4ac6f982094575836c623ce514fca2",
        "Sw360Id": "cf4ac6f982094575836c623ce514fca2",
        "CpeId": "",
        "Name": "clap",
        "Version": "4.5.60",
        "ComponentId": "f1a395ffa53b4d398e4d4d91f4550ea6",
        "ReleaseDate": "",
        "ExternalIds": {
            "package-url": "[\"pkg:cargo/clap_builder@4.5.60\",\"pkg:cargo/clap@4.5.60\"]"
        },
        "CreatedOn": "2026-05-10",
        "MainlineState": "SPECIFIC",
        "ClearingState": "APPROVED",
        "CreatedBy": "",
        "CreatorDepartment": "",
        "Languages": [],
        "DownloadUrl": "https://crates.io/api/v1/crates/clap/4.5.60/download",
        "Href": "",
        "SourceFile": "clap-4.5.60.zip",
        "SourceFileHash": "5848de2fe1fe81ea200744b4e2f03dc7a733ba04",
        "BinaryFile": "",
        "BinaryFileHash": "",
        "MapResult": "5-candidate-match-by-name"
    },
    {
        "Id": "978b6eb52bd1419dacd588226410fc35",
        "Sw360Id": "978b6eb52bd1419dacd588226410fc35",
        "CpeId": "",
        "Name": "clap",
        "Version": "4.5.40",
        "ComponentId": "f1a395ffa53b4d398e4d4d91f4550ea6",
        "ReleaseDate": "",
        "ExternalIds": {
            "package-url": "pkg:cargo/clap@4.5.40"
        },
        "CreatedOn": "2025-06-27",
        "MainlineState": "OPEN",
        "ClearingState": "NEW_CLEARING",
        "CreatedBy": "",
        "CreatorDepartment": "",
        "Languages": [],
        "DownloadUrl": "https://github.com/clap-rs/clap/archive/refs/tags/v4.5.40.zip",
        "Href": "",
        "SourceFile": "clap-4.5.40.zip",
        "SourceFileHash": "ddbd1895895266430bf0b5f7e588680a0a30b236",
        "BinaryFile": "",
        "BinaryFileHash": "",
        "MapResult": "5-candidate-match-by-name"
    },
    {
        "Id": "9da46e439f984b91ab21107bcf225556",
        "Sw360Id": "9da46e439f984b91ab21107bcf225556",
        "CpeId": "",
        "Name": "clap",
        "Version": "4.5.35",
        "ComponentId": "f1a395ffa53b4d398e4d4d91f4550ea6",
        "ReleaseDate": "",
        "ExternalIds": {
            "com.siemens.svm.component.id": "432033",
            "package-url": "pkg:cargo/clap@4.5.35"
        },
        "CreatedOn": "2025-04-07",
        "MainlineState": "SPECIFIC",
        "ClearingState": "APPROVED",
        "CreatedBy": "",
        "CreatorDepartment": "",
        "Languages": [],
        "DownloadUrl": "https://github.com/clap-rs/clap/archive/refs/tags/v4.5.35.zip",
        "Href": "",
        "SourceFile": "clap-4.5.35.zip",
        "SourceFileHash": "375bfe4dbe2cd4514078e3cf1331b2c9e887d0c1",
        "BinaryFile": "",
        "BinaryFileHash": "",
        "MapResult": "5-candidate-match-by-name"
    },
    {
        "Id": "55c064e4ad304719afb47d67cce4f7f2",
        "Sw360Id": "55c064e4ad304719afb47d67cce4f7f2",
        "CpeId": "",
        "Name": "clap",
        "Version": "4.5.34",
        "ComponentId": "f1a395ffa53b4d398e4d4d91f4550ea6",
        "ReleaseDate": "",
        "ExternalIds": {
            "package-url": "pkg:cargo/clap@4.5.34"
        },
        "CreatedOn": "2025-03-30",
        "MainlineState": "OPEN",
        "ClearingState": "NEW_CLEARING",
        "CreatedBy": "",
        "CreatorDepartment": "",
        "Languages": [],
        "DownloadUrl": "https://github.com/clap-rs/clap/archive/refs/tags/v4.5.34.zip",
        "Href": "",
        "SourceFile": "clap-4.5.34.zip",
        "SourceFileHash": "587b6d9410b35c9a77fed9e635071a4e8c797a70",
        "BinaryFile": "",
        "BinaryFileHash": "",
        "MapResult": "5-candidate-match-by-name"
    },
    {
        "Id": "1037bbd8e2fd4536a21796dc24494e0b",
        "Sw360Id": "1037bbd8e2fd4536a21796dc24494e0b",
        "CpeId": "",
        "Name": "clap",
        "Version": "4.5.32",
        "ComponentId": "f1a395ffa53b4d398e4d4d91f4550ea6",
        "ReleaseDate": "",
        "ExternalIds": {
            "package-url": "pkg:cargo/clap@4.5.32"
        },
        "CreatedOn": "2025-03-16",
        "MainlineState": "OPEN",
        "ClearingState": "NEW_CLEARING",
        "CreatedBy": "",
        "CreatorDepartment": "",
        "Languages": [],
        "DownloadUrl": "https://github.com/clap-rs/clap/archive/refs/tags/v4.5.32.zip",
        "Href": "",
        "SourceFile": "clap-4.5.32.zip",
        "SourceFileHash": "fc38ef6b40d31b26dba60ff749f4f1566e24137d",
        "BinaryFile": "",
        "BinaryFileHash": "",
        "MapResult": "5-candidate-match-by-name"
    },
    {
        "Id": "4be3cd7eff4e4cd98881335a543687f5",
        "Sw360Id": "4be3cd7eff4e4cd98881335a543687f5",
        "CpeId": "",
        "Name": "clap",
        "Version": "4.5.31",
        "ComponentId": "f1a395ffa53b4d398e4d4d91f4550ea6",
        "ReleaseDate": "",
        "ExternalIds": {
            "package-url": "pkg:cargo/clap@4.5.31"
        },
        "CreatedOn": "2025-03-02",
        "MainlineState": "OPEN",
        "ClearingState": "NEW_CLEARING",
        "CreatedBy": "",
        "CreatorDepartment": "",
        "Languages": [],
        "DownloadUrl": "https://github.com/clap-rs/clap/archive/refs/tags/v4.5.31.zip",
        "Href": "",
        "SourceFile": "clap-4.5.31.zip",
        "SourceFileHash": "b275f8df8a980fcc896a3c1c96512e294d98f585",
        "BinaryFile": "",
        "BinaryFileHash": "",
        "MapResult": "5-candidate-match-by-name"
    },
    {
        "Id": "8b759ab55f31449d80155e5e6cba0b73",
        "Sw360Id": "8b759ab55f31449d80155e5e6cba0b73",
        "CpeId": "",
        "Name": "clap",
        "Version": "4.5.30",
        "ComponentId": "f1a395ffa53b4d398e4d4d91f4550ea6",
        "ReleaseDate": "",
        "ExternalIds": {
            "package-url": "pkg:cargo/clap@4.5.30"
        },
        "CreatedOn": "2025-02-25",
        "MainlineState": "OPEN",
        "ClearingState": "NEW_CLEARING",
        "CreatedBy": "",
        "CreatorDepartment": "",
        "Languages": [],
        "DownloadUrl": "https://github.com/clap-rs/clap/archive/refs/tags/v4.5.30.zip",
        "Href": "",
        "SourceFile": "clap-4.5.30.zip",
        "SourceFileHash": "be010496293008eee380086f9de20b372c6249d4",
        "BinaryFile": "",
        "BinaryFileHash": "",
        "MapResult": "5-candidate-match-by-name"
    },
    {
        "Id": "ae3aafbed4aa497fb9f26b6732d33f40",
        "Sw360Id": "ae3aafbed4aa497fb9f26b6732d33f40",
        "CpeId": "",
        "Name": "clap",
        "Version": "4.5.29",
        "ComponentId": "f1a395ffa53b4d398e4d4d91f4550ea6",
        "ReleaseDate": "",
        "ExternalIds": {
            "package-url": "pkg:cargo/clap@4.5.29"
        },
        "CreatedOn": "2025-02-16",
        "MainlineState": "OPEN",
        "ClearingState": "NEW_CLEARING",
        "CreatedBy": "",
        "CreatorDepartment": "",
        "Languages": [],
        "DownloadUrl": "https://github.com/clap-rs/clap/archive/refs/tags/v4.5.29.zip",
        "Href": "",
        "SourceFile": "clap-4.5.29.zip",
        "SourceFileHash": "b424e231e95ce987829e3edb5e5ef3ccc614e526",
        "BinaryFile": "",
        "BinaryFileHash": "",
        "MapResult": "5-candidate-match-by-name"
    },
    {
        "Id": "deb0dd5edb2c4f5eb9427ad4cae460b6",
        "Sw360Id": "deb0dd5edb2c4f5eb9427ad4cae460b6",
        "CpeId": "",
        "Name": "clap",
        "Version": "4.5.28",
        "ComponentId": "f1a395ffa53b4d398e4d4d91f4550ea6",
        "ReleaseDate": "",
        "ExternalIds": {
            "package-url": "pkg:cargo/clap@4.5.28"
        },
        "CreatedOn": "2025-02-09",
        "MainlineState": "OPEN",
        "ClearingState": "NEW_CLEARING",
        "CreatedBy": "",
        "CreatorDepartment": "",
        "Languages": [],
        "DownloadUrl": "https://github.com/clap-rs/clap/archive/refs/tags/v4.5.28.zip",
        "Href": "",
        "SourceFile": "clap-4.5.28.zip",
        "SourceFileHash": "3e7c16cd11d59c857f87beaf02ee7f13477e0475",
        "BinaryFile": "",
        "BinaryFileHash": "",
        "MapResult": "5-candidate-match-by-name"
    },
    {
        "Id": "69b4bf070fff492c87890761f5ad9bba",
        "Sw360Id": "69b4bf070fff492c87890761f5ad9bba",
        "CpeId": "",
        "Name": "clap",
        "Version": "4.5.27",
        "ComponentId": "f1a395ffa53b4d398e4d4d91f4550ea6",
        "ReleaseDate": "",
        "ExternalIds": {
            "package-url": "pkg:cargo/clap@4.5.27"
        },
        "CreatedOn": "2025-01-26",
        "MainlineState": "OPEN",
        "ClearingState": "NEW_CLEARING",
        "CreatedBy": "",
        "CreatorDepartment": "",
        "Languages": [],
        "DownloadUrl": "https://github.com/clap-rs/clap/archive/refs/tags/v4.5.27.zip",
        "Href": "",
        "SourceFile": "clap-4.5.27.zip",
        "SourceFileHash": "756c62c072337d31329b76099f1f74500944bf2c",
        "BinaryFile": "",
        "BinaryFileHash": "",
        "MapResult": "5-candidate-match-by-name"
    },
    {
        "Id": "8f6a7acf59b44e81911a44066a401fc7",
        "Sw360Id": "8f6a7acf59b44e81911a44066a401fc7",
        "CpeId": "",
        "Name": "clap",
        "Version": "4.5.26",
        "ComponentId": "f1a395ffa53b4d398e4d4d91f4550ea6",
        "ReleaseDate": "",
        "ExternalIds": {
            "package-url": "pkg:cargo/clap@4.5.26"
        },
        "CreatedOn": "2025-01-20",
        "MainlineState": "OPEN",
        "ClearingState": "NEW_CLEARING",
        "CreatedBy": "",
        "CreatorDepartment": "",
        "Languages": [],
        "DownloadUrl": "https://github.com/clap-rs/clap/archive/refs/tags/v4.5.26.zip",
        "Href": "",
        "SourceFile": "clap-4.5.26.zip",
        "SourceFileHash": "b6fe828ce566fda7d5e78e4de65c756f2573aea3",
        "BinaryFile": "",
        "BinaryFileHash": "",
        "MapResult": "5-candidate-match-by-name"
    },
    {
        "Id": "8fe92a0e42ed4ae4bc4982db675ca3ed",
        "Sw360Id": "8fe92a0e42ed4ae4bc4982db675ca3ed",
        "CpeId": "",
        "Name": "clap",
        "Version": "4.5.23",
        "ComponentId": "f1a395ffa53b4d398e4d4d91f4550ea6",
        "ReleaseDate": "",
        "ExternalIds": {
            "package-url": "pkg:cargo/clap@4.5.23"
        },
        "CreatedOn": "2024-12-09",
        "MainlineState": "OPEN",
        "ClearingState": "NEW_CLEARING",
        "CreatedBy": "",
        "CreatorDepartment": "",
        "Languages": [],
        "DownloadUrl": "https://github.com/clap-rs/clap/archive/refs/tags/v4.5.23.zip",
        "Href": "",
        "SourceFile": "clap-4.5.23.zip",
        "SourceFileHash": "db00de552f488d5298eced4cc8aec30b5af84444",
        "BinaryFile": "",
        "BinaryFileHash": "",
        "MapResult": "5-candidate-match-by-name"
    },
    {
        "Id": "9f0a105f64e3449b8cfd702cb4b7d1e3",
        "Sw360Id": "9f0a105f64e3449b8cfd702cb4b7d1e3",
        "CpeId": "",
        "Name": "clap",
        "Version": "4.5.21",
        "ComponentId": "f1a395ffa53b4d398e4d4d91f4550ea6",
        "ReleaseDate": "",
        "ExternalIds": {
            "package-url": "pkg:cargo/clap@4.5.21"
        },
        "CreatedOn": "2024-11-17",
        "MainlineState": "OPEN",
        "ClearingState": "NEW_CLEARING",
        "CreatedBy": "",
        "CreatorDepartment": "",
        "Languages": [],
        "DownloadUrl": "https://github.com/clap-rs/clap/archive/refs/tags/v4.5.21.zip",
        "Href": "",
        "SourceFile": "clap-4.5.21.zip",
        "SourceFileHash": "4245aef69645ef57d7e887ef23bbf9e82c93b4fd",
        "BinaryFile": "",
        "BinaryFileHash": "",
        "MapResult": "5-candidate-match-by-name"
    },
    {
        "Id": "696ef670b48f4200b722d3a963613f5d",
        "Sw360Id": "696ef670b48f4200b722d3a963613f5d",
        "CpeId": "",
        "Name": "clap",
        "Version": "4.5.20",
        "ComponentId": "f1a395ffa53b4d398e4d4d91f4550ea6",
        "ReleaseDate": "",
        "ExternalIds": {
            "package-url": "pkg:cargo/clap@4.5.20"
        },
        "CreatedOn": "2024-10-28",
        "MainlineState": "OPEN",
        "ClearingState": "NEW_CLEARING",
        "CreatedBy": "",
        "CreatorDepartment": "",
        "Languages": [],
        "DownloadUrl": "https://github.com/clap-rs/clap/archive/refs/tags/v4.5.20.zip",
        "Href": "",
        "SourceFile": "clap-4.5.20.zip",
        "SourceFileHash": "68d79b609dfe1e60be4a6457b3785596d298db4b",
        "BinaryFile": "",
        "BinaryFileHash": "",
        "MapResult": "5-candidate-match-by-name"
    },
    {
        "Id": "d8ff4525be7a4fd2836bd4c0ccc9f96c",
        "Sw360Id": "d8ff4525be7a4fd2836bd4c0ccc9f96c",
        "CpeId": "",
        "Name": "clap",
        "Version": "4.1.7",
        "ComponentId": "f1a395ffa53b4d398e4d4d91f4550ea6",
        "ReleaseDate": "",
        "ExternalIds": {},
        "CreatedOn": "2023-05-03",
        "MainlineState": "OPEN",
        "ClearingState": "APPROVED",
        "CreatedBy": "",
        "CreatorDepartment": "",
        "Languages": [],
        "DownloadUrl": "https://github.com/clap-rs/clap/tree/v4.1.7",
        "Href": "",
        "SourceFile": "clap-4.1.7.zip",
        "SourceFileHash": "ff0834f3844c9c8c8b1121b504dc5cf550b8439d",
        "BinaryFile": "",
        "BinaryFileHash": "",
        "MapResult": "5-candidate-match-by-name"
    }
]

I reviewed the SW360 website and indeed, the component release cf4ac6f982094575836c623ce514fca2 has two entries for the pURL. And this make complex to address due of doubts:

• Should this fall directly into the SW360's maintainers jurisdiction ?
• Should we add a way to guarantee only one of the PuRL entries is selected?
• Should we avoid the entry entirely ? (taking in consideration that may had been already approved seems excessive)
• Should capycli bom validate search for this kind of occurrences ?

Edit

Well.. I failed to notice that the pURL entries at SW360 were forced to be have a List as a String . Probably this answers all the questions.

[tngraf]

Thanks for the report and the whole analysis.
Indeed, when we started using purls at SW360, we thought one purl per component/release would be enough. So we put this in the simple key/value store for external ids. Later we found out that there can be multiple purls per component/release. This can happen because the component is part of a large monorepo for multiple components, because of sources for different unix components in on SW360 entry, etc. At the end, we have this situation for years now.

But this is the first report, that CaPyCLI has problems with it.

[gernot-h]

Sorry for speaking up post-mortem, but I'd like to have a closer look on our options here. First of all, in the case of a good match when we don't create new components in the SBOM but just update existing ones, the code uses a different strategy, it will catch the error and silently ignore such PURL lists and NOT add them to the output BOM: https://github.com/sw360/capycli/blob/v2.11.1/capycli/bom/map_bom.py#L554. However, the try/catch block was missing in the other case.

So I think we should have both cases aligned. #219 now also implicitly changes the existing code path for updating an existing BOM entry as it will lead to the first PURL being used unconditionally.

Second, I think we should inform the user about the issue and neither silently drop the PURL list nor silently pick an arbitrary PURL.

In the past (or in other code paths?), I think I decided to actually generate an invalid SBOM by just setting the purl to the JSON string list and printed a warning about it informing the user (s)he has to fix it manually. So even if the user overlooks the warning, (s)he will stumble over the invalid field when processing the resulting BOM. However, I think nowerdays this doesn't work any more as the CycloneDX library forbids it.

[gernot-h]

By the way, @tngraf, this is an example of the logging strategy issue I recently mentioned. Even if we decide to print a warning here, chances are high that users will overlook it for large projects as there's so much output no-one reads in reality. (That's why I preferred to create an invalid SBOM in such cases in the past to force the user to have a look.) We already have colors which help a lot, but in some cases, there are also many irrelevant warnings. That's why I mentioned it would be great to have some kind of logging severity and filtering mechanisms.

[gernot-h]

The more I think about it, the more I'd suggest we don't pick an arbitrary PURL from the list, but just print a warning (containing the raw value from SW360) and keep the PURL field empty, which would be somehow aligned with the previous behaviour of not touching an existing PURL in case we can't parse the one from SW360. What do you think, @tngraf?

If I'm not mistaken, the code from #219 would also still crash in corner cases, like if an invalid PURL is read from SW360, but that's something we (I) can work on once we have a final decision how to handle it.

[rfuentess]

. Even if we decide to print a warning here, chances are high that users
will overlook it for large projects as there's so much output no-one reads
in reality. (That's why I preferred to create an invalid SBOM in such cases
in the past to force the user to have a look.) We already have colors which
help a lot, but in some cases, there are also many irrelevant warnings. T

The more I think about it, the more I'd suggest we don't pick an arbitrary
PURL from the list, but just print a warning (containing the raw value from
SW360) and keep the PURL field empty, which would be somehow aligned with
the previous behaviour of not touching an existing PURL in case we can't
parse the one from SW360.

I'd like to share my opinion after using CaPyCLI for the mapping, and
I agree with this approach.

CaPyCLI generates a new manifest with the mapped results. Depending
on the arguments provided by the user, we could end up with a scenario
where we have a mix of components: some with a single pURL, and others
where the pURL is now a list parsed as a string.

If the user also calls capycli bom update with this manifest, we risk
reversing components already validated in SW360.

In summary, this is why I would favor the option of making the entry
invalid, to enforce a human review. It may help them to realize,
just like me, that the argument --mode should had been used for
the mapping.