Skip to content

Commit 6dbd430

Browse files
tetronmr-c
authored andcommitted
Make check_exists cache negative results as well as positive ones
When loading a workflow with URL references that can't be resolved, don't keep trying them over and over again. Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <[email protected]>
1 parent b39d984 commit 6dbd430

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

schema_salad/fetcher.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -105,7 +105,10 @@ def fetch_text(self, url: str, content_types: Optional[List[str]] = None) -> str
105105
raise ValidationException(f"Unsupported scheme in url: {url}")
106106

107107
def check_exists(self, url: str) -> bool:
108-
if url in self.cache:
108+
entry = self.cache.get(url)
109+
if entry is False:
110+
return False
111+
elif entry is not None:
109112
return True
110113

111114
split = urllib.parse.urlsplit(url)
@@ -118,6 +121,7 @@ def check_exists(self, url: str) -> bool:
118121
resp = self.session.head(url, allow_redirects=True)
119122
resp.raise_for_status()
120123
except Exception:
124+
self.cache[url] = False
121125
return False
122126
self.cache[url] = True
123127
return True

0 commit comments

Comments
 (0)