fix: normalize SBOM purls to fix CVE mismatch in local eval

[felipecruz91]

This PR normalizes the packages in the SBOM attestation which is consumed during a local evaluation.

Notice that a large portion of the code in this PR such as NormalizeSBOM and DenormalizeSBOM among many other functions have been grabbed from the scout-cli-plugin repo.

[chrispatrick]

Looks ok to me, but I'd appreciate the eyes from @cdupuis

[cdupuis]

Could we move the copied bits to a place where we can use it from the scout-cli-plugin? Then we could delete the duplicate code in the other repo.

[felipecruz91]

@cdupuis Thanks for the feedback. I've moved the common bits to github.com/atomist-skills/go-skill/sbom/normalization in eed78d7. Happy to choose a different directory path that you think it could be clearer.

[rnorton5432]

Looks good! Thanks Felipe

[felipecruz91]

After running the skill locally using this payload.edn I identified an issue where the vulnerabilities query response was not being unmarshaled correctly. I've pushed a fix for that in this commit.

The no-fixable-packages-goal skill now returns a total of 27 deviations (CVEs with severity CRITICAL, HIGH, age > 30) when running the local evaluation for dockerscoutpolicy/gh-scout-demo-service:main (linux/amd64):

curl -X POST -H "Content-Type: application/edn" --data-binary @payload.edn http://localhost:8080

{:result[{:details{:fixedBy"4.17.3":purl"pkg:npm/[email protected]":severity"HIGH":vulnerability"CVE-2022-24999":cvssScore"7.5":locations[]}}{:details{:locations[]:fixedBy"3.3.3p1-r3":purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026distro=alpine-3.14.1":severity"HIGH":vulnerability"CVE-2022-0778":cvssScore"7.5"}}{:details{:cvssScore"7.5":locations[]:fixedBy"1.1.1n-r0":purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=openssl\u0026distro=alpine-3.14.1":severity"HIGH":vulnerability"CVE-2023-0215"}}{:details{:fixedBy"1.1.1n-r0":purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=openssl\u0026distro=alpine-3.14.1":severity"HIGH":vulnerability"CVE-2022-4304":cvssScore"7.5":locations[]}}{:details{:fixedBy"1.1.1n-r0":purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=openssl\u0026distro=alpine-3.14.1":severity"HIGH":vulnerability"CVE-2021-3712":cvssScore"7.5":locations[]}}{:details{:locations[]:fixedBy"1.1.1n-r0":purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=openssl\u0026distro=alpine-3.14.1":severity"HIGH":vulnerability"CVE-2022-4450":cvssScore"7.5"}}{:details{:fixedBy"1.1.1n-r0":purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=openssl\u0026distro=alpine-3.14.1":severity"HIGH":vulnerability"CVE-2023-0464":cvssScore"7.5":locations[]}}{:details{:cvssScore"7.5":locations[]:fixedBy"1.1.1n-r0":purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=openssl\u0026distro=alpine-3.14.1":severity"HIGH":vulnerability"CVE-2021-3711"}}{:details{:severity"HIGH":vulnerability"CVE-2023-0465":cvssScore"7.5":locations[]:fixedBy"1.1.1n-r0":purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=openssl\u0026distro=alpine-3.14.1"}}{:details{:fixedBy"1.1.1n-r0":purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=openssl\u0026distro=alpine-3.14.1":severity"HIGH":vulnerability"CVE-2023-0286":cvssScore"7.5":locations[]}}{:details{:purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=openssl\u0026distro=alpine-3.14.1":severity"HIGH":vulnerability"CVE-2022-2097":cvssScore"7.5":locations[]:fixedBy"1.1.1n-r0"}}{:details{:fixedBy"1.1.1n-r0":purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=openssl\u0026distro=alpine-3.14.1":severity"HIGH":vulnerability"CVE-2022-0778":cvssScore"7.5":locations[]}}{:details{:fixedBy"6.7.3":purl"pkg:npm/[email protected]":severity"HIGH":vulnerability"CVE-2022-24999":cvssScore"7.5":locations[]}}{:details{:locations[]:fixedBy"1.33.1-r6":purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=busybox\u0026distro=alpine-3.14.1":severity"HIGH":vulnerability"CVE-2022-28391":cvssScore"7.2"}}{:details{:purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=busybox\u0026distro=alpine-3.14.1":severity"HIGH":vulnerability"CVE-2021-42383":cvssScore"7.2":locations[]:fixedBy"1.33.1-r6"}}{:details{:fixedBy"1.33.1-r6":purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=busybox\u0026distro=alpine-3.14.1":severity"HIGH":vulnerability"CVE-2021-42374":cvssScore"7.2":locations[]}}{:details{:severity"HIGH":vulnerability"CVE-2021-42379":cvssScore"7.2":locations[]:fixedBy"1.33.1-r6":purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=busybox\u0026distro=alpine-3.14.1"}}{:details{:severity"HIGH":vulnerability"CVE-2021-42386":cvssScore"7.2":locations[]:fixedBy"1.33.1-r6":purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=busybox\u0026distro=alpine-3.14.1"}}{:details{:purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=busybox\u0026distro=alpine-3.14.1":severity"HIGH":vulnerability"CVE-2021-42378":cvssScore"7.2":locations[]:fixedBy"1.33.1-r6"}}{:details{:fixedBy"1.33.1-r6":purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=busybox\u0026distro=alpine-3.14.1":severity"HIGH":vulnerability"CVE-2021-42385":cvssScore"7.2":locations[]}}{:details{:fixedBy"1.33.1-r6":purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=busybox\u0026distro=alpine-3.14.1":severity"HIGH":vulnerability"CVE-2021-42384":cvssScore"7.2":locations[]}}{:details{:purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=busybox\u0026distro=alpine-3.14.1":severity"HIGH":vulnerability"CVE-2021-42381":cvssScore"7.2":locations[]:fixedBy"1.33.1-r6"}}{:details{:fixedBy"1.33.1-r6":purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=busybox\u0026distro=alpine-3.14.1":severity"HIGH":vulnerability"CVE-2021-42382":cvssScore"7.2":locations[]}}{:details{:fixedBy"1.33.1-r6":purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=busybox\u0026distro=alpine-3.14.1":severity"HIGH":vulnerability"CVE-2021-42375":cvssScore"7.2":locations[]}}{:details{:vulnerability"ALPINE-13661":cvssScore"7.2":locations[]:fixedBy"1.33.1-r6":purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=busybox\u0026distro=alpine-3.14.1":severity"HIGH"}}{:details{:purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=busybox\u0026distro=alpine-3.14.1":severity"HIGH":vulnerability"CVE-2021-42380":cvssScore"7.2":locations[]:fixedBy"1.33.1-r6"}}{:details{:severity"HIGH":vulnerability"CVE-2022-37434":cvssScore"7.5":locations[]:fixedBy"1.2.12-r0":purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026distro=alpine-3.14.1"}}{:details{:fixedBy"1.2.12-r0":purl"pkg:apk/alpine/[email protected]?arch=x86_64\u0026distro=alpine-3.14.1":severity"HIGH":vulnerability"CVE-2018-25032":cvssScore"7.5":locations[]}}]}%  

However, the Scout website reports 20 deviations (2C, 18H).

Any ideas?

[felipecruz91]

Regarding the mismatch count in the comment above, I've pushed a fix in 2ddeb03.

Now I get exactly 20 deviations (2C 18H). See the evidence below:

{:result [{:details {:vulnerability "CVE-2022-24999" :cvssScore "7.5" :locations [] :fixedBy "4.17.3" :purl "pkg:npm/[email protected]" :severity "HIGH"}} {:details {:fixedBy "3.3.3p1-r3" :purl "pkg:apk/alpine/[email protected]?arch=x86_64\u0026distro=alpine-3.14.1" :severity "HIGH" :vulnerability "CVE-2022-0778" :cvssScore "7.5" :locations []}} {:details {:fixedBy "1.1.1l-r0" :purl "pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=openssl\u0026distro=alpine-3.14.1" :severity "HIGH" :vulnerability "CVE-2021-3712" :cvssScore "7.4" :locations []}} {:details {:fixedBy "1.1.1t-r1" :purl "pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=openssl\u0026distro=alpine-3.14.1" :severity "HIGH" :vulnerability "CVE-2023-0464" :cvssScore "7.5" :locations []}} {:details {:fixedBy "1.1.1l-r0" :purl "pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=openssl\u0026distro=alpine-3.14.1" :severity "CRITICAL" :vulnerability "CVE-2021-3711" :cvssScore "9.8" :locations []}} {:details {:locations [] :fixedBy "1.1.1t-r0" :purl "pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=openssl\u0026distro=alpine-3.14.1" :severity "HIGH" :vulnerability "CVE-2023-0286" :cvssScore "0.0"}} {:details {:severity "HIGH" :vulnerability "CVE-2022-0778" :cvssScore "7.5" :locations [] :fixedBy "1.1.1n-r0" :purl "pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=openssl\u0026distro=alpine-3.14.1"}} {:details {:fixedBy "6.7.3" :purl "pkg:npm/[email protected]" :severity "HIGH" :vulnerability "CVE-2022-24999" :cvssScore "7.5" :locations []}} {:details {:fixedBy "1.33.1-r7" :purl "pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=busybox\u0026distro=alpine-3.14.1" :severity "HIGH" :vulnerability "CVE-2022-28391" :cvssScore "8.8" :locations []}} {:details {:fixedBy "1.33.1-r6" :purl "pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=busybox\u0026distro=alpine-3.14.1" :severity "HIGH" :vulnerability "CVE-2021-42383" :cvssScore "7.2" :locations []}} {:details {:cvssScore "7.2" :locations [] :fixedBy "1.33.1-r6" :purl "pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=busybox\u0026distro=alpine-3.14.1" :severity "HIGH" :vulnerability "CVE-2021-42379"}} {:details {:cvssScore "7.2" :locations [] :fixedBy "1.33.1-r6" :purl "pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=busybox\u0026distro=alpine-3.14.1" :severity "HIGH" :vulnerability "CVE-2021-42386"}} {:details {:purl "pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=busybox\u0026distro=alpine-3.14.1" :severity "HIGH" :vulnerability "CVE-2021-42378" :cvssScore "7.2" :locations [] :fixedBy "1.33.1-r6"}} {:details {:fixedBy "1.33.1-r6" :purl "pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=busybox\u0026distro=alpine-3.14.1" :severity "HIGH" :vulnerability "CVE-2021-42385" :cvssScore "7.2" :locations []}} {:details {:cvssScore "7.2" :locations [] :fixedBy "1.33.1-r6" :purl "pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=busybox\u0026distro=alpine-3.14.1" :severity "HIGH" :vulnerability "CVE-2021-42384"}} {:details {:severity "HIGH" :vulnerability "CVE-2021-42381" :cvssScore "7.2" :locations [] :fixedBy "1.33.1-r6" :purl "pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=busybox\u0026distro=alpine-3.14.1"}} {:details {:vulnerability "CVE-2021-42382" :cvssScore "7.2" :locations [] :fixedBy "1.33.1-r6" :purl "pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=busybox\u0026distro=alpine-3.14.1" :severity "HIGH"}} {:details {:fixedBy "1.33.1-r6" :purl "pkg:apk/alpine/[email protected]?arch=x86_64\u0026upstream=busybox\u0026distro=alpine-3.14.1" :severity "HIGH" :vulnerability "CVE-2021-42380" :cvssScore "7.2" :locations []}} {:details {:vulnerability "CVE-2022-37434" :cvssScore "9.8" :locations [] :fixedBy "1.2.12-r2" :purl "pkg:apk/alpine/[email protected]?arch=x86_64\u0026distro=alpine-3.14.1" :severity "CRITICAL"}} {:details {:fixedBy "1.2.12-r0" :purl "pkg:apk/alpine/[email protected]?arch=x86_64\u0026distro=alpine-3.14.1" :severity "HIGH" :vulnerability "CVE-2018-25032" :cvssScore "7.5" :locations []}}]}