Skip to content

Conversation

@ziadhany
Copy link
Collaborator

@ziadhany ziadhany changed the title Add initial migration to Gentoo importer v2 Migrate Gentoo importer to advisory V2 Dec 30, 2025
@ziadhany ziadhany marked this pull request as ready for review January 1, 2026 13:58
@ziadhany
Copy link
Collaborator Author

ziadhany commented Jan 1, 2026

gentoo importer logs:

Importing data using gentoo_importer_v2
INFO 2026-01-01 13:56:44.096374 UTC Pipeline [GentooImporterPipeline] starting
INFO 2026-01-01 13:56:44.096532 UTC Step [clone] starting
INFO 2026-01-01 13:56:44.096601 UTC Cloning `git+https://anongit.gentoo.org/git/data/glsa.git`
INFO 2026-01-01 13:57:10.664048 UTC Step [clone] completed in 27 seconds
INFO 2026-01-01 13:57:10.664196 UTC Step [collect_and_store_advisories] starting
INFO 2026-01-01 13:57:10.692323 UTC Collecting 3,809 advisories
INFO 2026-01-01 13:57:15.280573 UTC Progress: 10% (381/3809) ETA: 41 seconds
INFO 2026-01-01 13:57:21.279172 UTC Progress: 20% (762/3809) ETA: 42 seconds
INFO 2026-01-01 13:57:27.515311 UTC Progress: 30% (1143/3809) ETA: 39 seconds
InvalidVersion - version: 3.24.48:3 - error:'3.24.48:3' is not a valid <class 'univers.versions.GentooVersion'>
INFO 2026-01-01 13:57:34.218837 UTC Progress: 40% (1524/3809) ETA: 35 seconds
INFO 2026-01-01 13:57:41.346657 UTC Progress: 50% (1905/3809) ETA: 31 seconds
INFO 2026-01-01 13:57:46.184409 UTC Progress: 60% (2286/3809) ETA: 24 seconds
InvalidVersion - version: 6.9.3:6 - error:'6.9.3:6' is not a valid <class 'univers.versions.GentooVersion'>
INFO 2026-01-01 13:57:50.800820 UTC Progress: 70% (2667/3809) ETA: 17 seconds
INFO 2026-01-01 13:57:58.011454 UTC Progress: 80% (3048/3809) ETA: 12 seconds
INFO 2026-01-01 13:58:03.875429 UTC Progress: 90% (3429/3809) ETA: 6 seconds
INFO 2026-01-01 13:58:09.804842 UTC Progress: 100% (3809/3809)
INFO 2026-01-01 13:58:09.823755 UTC Successfully collected 3,809 advisories
INFO 2026-01-01 13:58:09.823893 UTC Step [collect_and_store_advisories] completed in 59 seconds
INFO 2026-01-01 13:58:09.823966 UTC Step [clean_downloads] starting
INFO 2026-01-01 13:58:09.824028 UTC Removing cloned repository
INFO 2026-01-01 13:58:09.890193 UTC Step [clean_downloads] completed in 0 seconds
INFO 2026-01-01 13:58:09.890356 UTC Pipeline completed in 86 seconds (1.4 minutes)

from vulnerabilities.models import AdvisoryV2
from django.db.models import Count
duplicates = (
    AdvisoryV2.objects
    .values('avid')
    .annotate(count=Count('id'))
    .filter(count__gt=1)
)
len(duplicates)
Out[2]: 0
AdvisoryV2.objects.count()
Out[3]: 3809

Comment on lines +15 to +18
<unaffected range="ge">1.9.7</unaffected>
<unaffected range="rgt">1.8.18</unaffected>
<vulnerable range="lt">1.9.7</vulnerable>
<vulnerable range="eq">0.1.1</vulnerable>
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to fix this to ensure affected_packages is accurate.

@ziadhany
Copy link
Collaborator Author

ziadhany commented Jan 1, 2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant