Skip to content

feat: rework combine-to-osv to begin to combine NVD and CVE5 OSV records #4086

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 93 commits into from
Oct 7, 2025
Merged

feat: rework combine-to-osv to begin to combine NVD and CVE5 OSV records #4086

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
93 commits
Select commit Hold shift + click to select a range
1a466f1
Fix missing delete
jess-lowe Aug 28, 2025
f5b40e4
Remove Linux special treatment
jess-lowe Aug 29, 2025
bad798b
Improve inverse version extraction
jess-lowe Aug 29, 2025
b62ceb3
Fix double commit hash sections
jess-lowe Aug 29, 2025
86ace30
Merge remote-tracking branch 'upstream/master' into feat/cve-conv-cro…
jess-lowe Aug 29, 2025
b4f5b57
Remove database_specific map from initial vuln instance
jess-lowe Sep 1, 2025
c352ac1
fix version_extraction test
jess-lowe Sep 1, 2025
f17b8c5
rename and repackage cvelist converstion
jess-lowe Sep 1, 2025
0638dff
repackage
jess-lowe Sep 1, 2025
f7b0de3
cvelist mass converter script
jess-lowe Sep 1, 2025
0fc040d
Remove outcomes logic for now (not very concurrency-friendly)
jess-lowe Sep 1, 2025
227ba92
Converter script
jess-lowe Sep 1, 2025
e2f5b64
rename files back
jess-lowe Sep 1, 2025
c56e9d5
dockerfile and cron job
jess-lowe Sep 2, 2025
7df4e7c
Merge remote-tracking branch 'upstream/master' into feat/cve-conv-cro…
jess-lowe Sep 4, 2025
cf40025
flatten if statements
jess-lowe Sep 4, 2025
394d039
Revert "dockerfile and cron job"
jess-lowe Sep 8, 2025
34e3505
Merge remote-tracking branch 'upstream/master' into feat/cve-conv-cro…
jess-lowe Sep 8, 2025
013c5c8
update logger
jess-lowe Sep 8, 2025
34878fb
fix lint through refactoring everything :(
jess-lowe Sep 8, 2025
af05357
Added flags and removed double parsing
jess-lowe Sep 9, 2025
9b2ce86
rename sortBadSemver
jess-lowe Sep 9, 2025
68f8d55
refactored parts for clarity
jess-lowe Sep 9, 2025
28e0691
deal with if number of parts are not 2 or 3
jess-lowe Sep 9, 2025
e69b7fb
rename sortBadSemver in tests
jess-lowe Sep 9, 2025
0114a27
Refactor VersionToCommit to ONLY return a commit, not the AffectedCom…
jess-lowe Sep 9, 2025
5fa7c47
update test snapshots
jess-lowe Sep 9, 2025
fbdd60f
FIX LINT
jess-lowe Sep 9, 2025
56c6db5
MUCH PRETTIER CODE
jess-lowe Sep 9, 2025
40b1d9f
Merge remote-tracking branch 'upstream/master' into feat/cve-git-reso…
jess-lowe Sep 9, 2025
3c528d7
Merge branch 'refactor/versions-to-commit' into feat/cve-git-resolve-1
jess-lowe Sep 9, 2025
d36ada9
Enable git commit extraction in affected field extracted vulns.
jess-lowe Sep 10, 2025
650290d
Save unresolved version ranges in database_specific
jess-lowe Sep 10, 2025
77eab2a
fix the concurrency issue with Metrics.Notes
jess-lowe Sep 10, 2025
e8d344a
Fix testcases
jess-lowe Sep 12, 2025
18ca577
Merge remote-tracking branch 'upstream/master' into feat/cve-git-reso…
jess-lowe Sep 12, 2025
7641ad1
fix logging issues
jess-lowe Sep 12, 2025
da21f5d
fix lint
jess-lowe Sep 12, 2025
49299c0
fix folder name
jess-lowe Sep 12, 2025
89b4541
improve naming clarity
jess-lowe Sep 15, 2025
667d6cf
update source for affected
jess-lowe Sep 15, 2025
ac0a979
fix lint
jess-lowe Sep 16, 2025
b9909a7
wrong place
jess-lowe Sep 16, 2025
6d0fe8c
merge decoupled alpine and debian
jess-lowe Sep 17, 2025
13b8a75
Merge remote-tracking branch 'upstream/master' into feat/combine-2-osv
jess-lowe Sep 17, 2025
c984775
Start combining data naively
jess-lowe Sep 18, 2025
b5a7303
remove unneeded bits
jess-lowe Sep 18, 2025
459eb95
attempt to Parse oneliner range
jess-lowe Sep 18, 2025
f975c3b
fix lint
jess-lowe Sep 18, 2025
2fa02e7
Merge branch 'feat/cve-git-resolve-1' into feat/combine-2-osv
jess-lowe Sep 19, 2025
60ad54b
Merge branch 'master' into feat/cve-git-resolve-1
jess-lowe Sep 19, 2025
24d71de
Treat version ranges git resolution as a stack to reduce chances of d…
jess-lowe Sep 19, 2025
8be73f6
Assume if only one version that value is LAST AFFECTED not fixed
jess-lowe Sep 19, 2025
208f809
refactor some duplicate code and fix tests
jess-lowe Sep 21, 2025
31bdcab
fix lint
jess-lowe Sep 21, 2025
9a881ee
Merge branch 'feat/cve-git-resolve-1' into feat/combine-2-osv
jess-lowe Sep 22, 2025
7525568
add reading debian bucket
jess-lowe Sep 22, 2025
454d73f
Begin affected ranges comparisons
jess-lowe Sep 23, 2025
1996520
make buildVersionRange a helper function
jess-lowe Sep 26, 2025
542f764
Add some base logic for combining cve5 and nvd
jess-lowe Sep 26, 2025
8ae771a
Improve affected ranges combining
jess-lowe Sep 28, 2025
bdffd05
use different globbing pattern
jess-lowe Sep 29, 2025
389349d
Merge branch 'master' into feat/cve-git-resolve-1
jess-lowe Sep 29, 2025
eca80e8
Remove duplicate code
jess-lowe Sep 29, 2025
6cf056c
Improve pathwalking and improved tests
jess-lowe Oct 1, 2025
360883d
Merge branch 'master' into feat/cve-git-resolve-1
jess-lowe Oct 1, 2025
0ea46b7
Make clearer where passing as a pointer
jess-lowe Oct 1, 2025
23e8ca3
refactor gitToCommits
jess-lowe Oct 1, 2025
62ddc33
fix lint
jess-lowe Oct 1, 2025
ff12127
Change output path for nvd osv files
jess-lowe Oct 1, 2025
9f63019
Merge branch 'feat/cve-git-resolve-1' into feat/combine-2-osv
jess-lowe Oct 1, 2025
454d1da
Remove unused functions
jess-lowe Oct 1, 2025
93728c7
fix merge conflict
jess-lowe Oct 1, 2025
c1ce5b8
unstage buildVersionRange refactor
jess-lowe Oct 1, 2025
6d3934f
update mod file
jess-lowe Oct 1, 2025
e3ff1b8
Fix buildVersionRange accessibility
jess-lowe Oct 1, 2025
a5fa4ed
Merge remote-tracking branch 'upstream/master' into feat/combine-2-osv
jess-lowe Oct 1, 2025
5d3f2c0
Merge branch 'master' into feat/combine-2-osv
jess-lowe Oct 1, 2025
9a393eb
Merge branch 'master' into feat/combine-2-osv
jess-lowe Oct 1, 2025
e871bae
add cna check back
jess-lowe Oct 1, 2025
2c378d2
fix lint
jess-lowe Oct 1, 2025
5190343
address listbucket file name things
jess-lowe Oct 2, 2025
1309110
move some code and add docstrings
jess-lowe Oct 2, 2025
0106f3a
Added comment for bucket objs
jess-lowe Oct 2, 2025
70ea025
combeinTwoOSV records as its own function
jess-lowe Oct 2, 2025
6347b93
refactor repo map loop
jess-lowe Oct 2, 2025
e7530ae
panic dont fatal
jess-lowe Oct 2, 2025
37c1f93
Added more testcases and fixed a bug.
jess-lowe Oct 2, 2025
bc7215c
Added a test for combining two records
jess-lowe Oct 2, 2025
3b86c16
fix lint
jess-lowe Oct 2, 2025
89df10c
fix name
jess-lowe Oct 2, 2025
8849b7a
address nits
jess-lowe Oct 3, 2025
5b04ec5
fix the output path comment
jess-lowe Oct 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion deployment/clouddeploy/gke-workers/environments/oss-vdb-test/nvd-cve-osv.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,4 @@ spec:
- name: CPEREPO_GCS_PATH
value: gs://osv-test-cve-osv-conversion/cpe_repos/cpe_product_to_repo.json
- name: OSV_OUTPUT_GCS_PATH
value: gs://osv-test-cve-osv-conversion/parts/nvd
value: gs://osv-test-cve-osv-conversion/nvd-osv
2 changes: 1 addition & 1 deletion deployment/clouddeploy/gke-workers/environments/oss-vdb/nvd-cve-osv.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,4 @@ spec:
- name: CPEREPO_GCS_PATH
value: gs://cve-osv-conversion/cpe_repos/cpe_product_to_repo.json
- name: OSV_OUTPUT_GCS_PATH
value: gs://cve-osv-conversion/parts/nvd
value: gs://cve-osv-conversion/nvd-osv
Loading
Loading