Skip to content

Commit 30bc341

Browse files
authored
fix: limit number of search indices added to Bugs (#4285)
[openSUSE-SU-2024:12948-1](https://osv.dev/vulnerability/openSUSE-SU-2024:12948-1) has so many upstream vulns that it made >6500 search_indices, which (combined with the other fields) was too many indexed properties for datastore to handle. Put a limit on the number of search indices populated from alias/upstreams to avoid this problem.
1 parent 35486c9 commit 30bc341

File tree

1 file changed

+10
-3
lines changed

1 file changed

+10
-3
lines changed

osv/models.py

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -420,13 +420,15 @@ def _pre_put_hook(self): # pylint: disable=arguments-differ
420420
for ecosystem in self.ecosystem:
421421
search_indices.update(_tokenize(ecosystem))
422422

423+
extra_search_indices = set()
424+
423425
for alias in self.aliases:
424-
search_indices.update(_tokenize(alias))
426+
extra_search_indices.update(_tokenize(alias))
425427

426428
# Please note this will not include exhaustive transitive upstream
427429
# so may not appear for all cases.
428430
for upstream in self.upstream_raw:
429-
search_indices.update(_tokenize(upstream))
431+
extra_search_indices.update(_tokenize(upstream))
430432

431433
for affected_package in self.affected_packages:
432434
for affected_range in affected_package.ranges:
@@ -437,7 +439,12 @@ def _pre_put_hook(self): # pylint: disable=arguments-differ
437439
repo_url_indices.append(url_no_https) # add url without https://
438440
search_indices.update(repo_url_indices)
439441

440-
self.search_indices = list(set(search_indices))
442+
# Excessive amounts of search indices cause the record to have too many
443+
# indexed properties to store in the database. Arbitrarily limit the number
444+
# of search indices from aliases/upstream to curb this.
445+
# TODO(michaelkedar): handle this more gracefully.
446+
search_indices.update(list(extra_search_indices)[:5000])
447+
self.search_indices = list(search_indices)
441448
self.search_indices.sort()
442449

443450
search_tags = set()

0 commit comments

Comments
 (0)