Skip to content

Commit 31d0117

Browse files
tetronmr-c
authored andcommitted
Make check_exists cache negative results as well as positive ones
When loading a workflow with URL references that can't be resolved, don't keep trying them over and over again. Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <[email protected]>
1 parent 097061d commit 31d0117

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

schema_salad/fetcher.py

+5-1
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,10 @@ def fetch_text(self, url: str, content_types: Optional[List[str]] = None) -> str
108108
raise ValidationException(f"Unsupported scheme in url: {url}")
109109

110110
def check_exists(self, url: str) -> bool:
111-
if url in self.cache:
111+
entry = self.cache.get(url)
112+
if entry is False:
113+
return False
114+
elif entry is not None:
112115
return True
113116

114117
split = urllib.parse.urlsplit(url)
@@ -121,6 +124,7 @@ def check_exists(self, url: str) -> bool:
121124
resp = self.session.head(url, allow_redirects=True)
122125
resp.raise_for_status()
123126
except Exception:
127+
self.cache[url] = False
124128
return False
125129
self.cache[url] = True
126130
return True

0 commit comments

Comments
 (0)