Extract nested CycloneDX components and deduplicate PURLs during mirror SBOM ingestion

[abhinavgautam01]

Summary

Extend mirror SBOM ingestion to collect PURLs from nested CycloneDX components and the metadata component and deduplicate extracted PURLs for both CycloneDX and SPDX before dispatching mirror work.

The current flat extraction can omit packages represented in a component hierarchy. Repeated PURLs also produce redundant mirror jobs and misleading package totals.

Affected code

• internal/mirror/source.go: CycloneDX and SPDX PURL extraction
• internal/mirror/mirror.go: creation and dispatch of mirror jobs
• internal/mirror/source_test.go: SBOM parsing coverage

Current behavior

CycloneDX extraction scans the top-level bom.Components collection and appends each nonempty PackageURL. It does not walk child components or inspect bom.Metadata.Component. A document whose relevant packages appear only in those locations can therefore yield an incomplete set of mirror inputs.

Neither SBOM extraction path consistently deduplicates the resulting PURL list. If a package appears more than once, the mirror worker pool can schedule multiple jobs for the same package, repeating resolution and potentially artifact requests while counting the package multiple times.

This is an ingestion-coverage and efficiency enhancement. SPDX's package list is a separate representation: the recursive traversal proposed here applies to CycloneDX component trees, while PURL deduplication applies to both formats.

Expected behavior

• Collect nonempty PURLs from top-level CycloneDX components and recursively from their child components.
• Inspect metadata.component, including its descendants, even when the top-level component list is absent.
• Traverse descendants even when their parent has no PURL.
• Deduplicate PURLs extracted from either CycloneDX or SPDX before creating mirror jobs.
• Preserve deterministic first-seen ordering so input ordering and progress reporting remain predictable.
• Preserve existing handling of malformed and unsupported PURLs; collecting a component does not imply that its ecosystem or artifact is mirrorable.

Suggested implementation

Use a shared ordered collector backed by a map[string]struct{}. For CycloneDX, visit each component's PURL and then its children and also visit the metadata component when present. For SPDX, send each extracted package PURL through the same deduplication collector.

Exact-string deduplication is a reasonable initial scope. If semantically equivalent PURL spellings are also deduplicated, use the project's existing canonicalization rules rather than ad hoc lowercasing: case sensitivity varies by ecosystem.

Do not treat CycloneDX dependency-graph references as PURLs. This proposal concerns component objects that actually carry PURLs, including components nested beneath other components.

Acceptance tests

• Parse a CycloneDX document containing multiple levels of nested components.
• Parse a document with only a metadata component and no top-level components.
• Verify children are collected when their parent has no PURL.
• Repeat the same PURL at the root, within a child and in the metadata component; verify it appears only once.
• Repeat a PURL across SPDX package references; verify it appears only once.
• Verify stable ordering and unchanged behavior for ordinary flat SBOMs.
• Verify mirror totals and scheduled work reflect the deduplicated inputs.