Skip to content

Commit c673535

Browse files
committed
Make check_exists cache negative results as well as positive ones
When loading a workflow with URL references that can't be resolved, don't keep trying them over and over again. Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <[email protected]>
1 parent 2a5a0fc commit c673535

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

schema_salad/fetcher.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -95,7 +95,10 @@ def fetch_text(self, url: str, content_types: Optional[List[str]] = None) -> str
9595
raise ValidationException(f"Unsupported scheme in url: {url}")
9696

9797
def check_exists(self, url: str) -> bool:
98-
if url in self.cache:
98+
entry = self.cache.get(url)
99+
if entry is False:
100+
return False
101+
elif entry is not None:
99102
return True
100103

101104
split = urllib.parse.urlsplit(url)
@@ -108,6 +111,7 @@ def check_exists(self, url: str) -> bool:
108111
resp = self.session.head(url, allow_redirects=True)
109112
resp.raise_for_status()
110113
except Exception:
114+
self.cache[url] = False
111115
return False
112116
self.cache[url] = True
113117
return True

0 commit comments

Comments
 (0)