Skip to content

Save which tomograms have processing job ids so they don't get rechecked every time #504

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
stephen-riggs merged 1 commit into from
Mar 11, 2025
Merged

Save which tomograms have processing job ids so they don't get rechecked every time #504

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 53 additions & 50 deletions src/murfey/client/contexts/tomo.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@
super().__init__("Tomography", acquisition_software)
self._basepath = basepath
self._tilt_series: Dict[str, List[Path]] = {}
self._tilt_series_with_pjids: List[str] = []
self._tilt_series_sizes: Dict[str, int] = {}
self._completed_tilt_series: List[str] = []
self._aligned_tilt_series: List[str] = []
Expand Down Expand Up @@ -116,57 +117,59 @@
capture_post(dcg_url, json=dcg_data)

for tilt_series in self._tilt_series.keys():
dc_url = f"{str(environment.url.geturl())}/visits/{environment.visit}/{environment.murfey_session}/start_data_collection"
dc_data = {
"experiment_type": "tomography",
"file_extension": file_extension,
"acquisition_software": self._acquisition_software,
"image_directory": image_directory,
"data_collection_tag": tilt_series,
"source": str(self._basepath),
"tag": tilt_series,
}
if (
environment.data_collection_parameters
and environment.data_collection_parameters.get("voltage")
):
# Once mdoc parameters are known register processing jobs
dc_data.update(
{
"voltage": environment.data_collection_parameters[
"voltage"
],
"pixel_size_on_image": environment.data_collection_parameters[
"pixel_size_on_image"
],
"image_size_x": environment.data_collection_parameters[
"image_size_x"
],
"image_size_y": environment.data_collection_parameters[
"image_size_y"
],
"magnification": environment.data_collection_parameters[
"magnification"
],
}
)
capture_post(dc_url, json=dc_data)

proc_url = f"{str(environment.url.geturl())}/visits/{environment.visit}/{environment.murfey_session}/register_processing_job"
for recipe in ("em-tomo-preprocess", "em-tomo-align"):
capture_post(
proc_url,
json={
"tag": tilt_series,
"source": str(self._basepath),
"recipe": recipe,
"experiment_type": "tomography",
},
if tilt_series not in self._tilt_series_with_pjids:
dc_url = f"{str(environment.url.geturl())}/visits/{environment.visit}/{environment.murfey_session}/start_data_collection"
dc_data = {
"experiment_type": "tomography",
"file_extension": file_extension,
"acquisition_software": self._acquisition_software,
"image_directory": image_directory,
"data_collection_tag": tilt_series,
"source": str(self._basepath),
"tag": tilt_series,
}
if (
environment.data_collection_parameters
and environment.data_collection_parameters.get("voltage")
):
# Once mdoc parameters are known register processing jobs
dc_data.update(

Check warning on line 136 in src/murfey/client/contexts/tomo.py

View check run for this annotation

Codecov / codecov/patch

src/murfey/client/contexts/tomo.py#L136 

Added line #L136 was not covered by tests
{
"voltage": environment.data_collection_parameters[
"voltage"
],
"pixel_size_on_image": environment.data_collection_parameters[
"pixel_size_on_image"
],
"image_size_x": environment.data_collection_parameters[
"image_size_x"
],
"image_size_y": environment.data_collection_parameters[
"image_size_y"
],
"magnification": environment.data_collection_parameters[
"magnification"
],
}
)
capture_post(dc_url, json=dc_data)

Check warning on line 155 in src/murfey/client/contexts/tomo.py

View check run for this annotation

Codecov / codecov/patch

src/murfey/client/contexts/tomo.py#L155 

Added line #L155 was not covered by tests

proc_url = f"{str(environment.url.geturl())}/visits/{environment.visit}/{environment.murfey_session}/register_processing_job"

Check warning on line 157 in src/murfey/client/contexts/tomo.py

View check run for this annotation

Codecov / codecov/patch

src/murfey/client/contexts/tomo.py#L157 

Added line #L157 was not covered by tests
for recipe in ("em-tomo-preprocess", "em-tomo-align"):
capture_post(

Check warning on line 159 in src/murfey/client/contexts/tomo.py

View check run for this annotation

Codecov / codecov/patch

src/murfey/client/contexts/tomo.py#L159 

Added line #L159 was not covered by tests
proc_url,
json={
"tag": tilt_series,
"source": str(self._basepath),
"recipe": recipe,
"experiment_type": "tomography",
},
)
self._tilt_series_with_pjids.append(tilt_series)

Check warning on line 168 in src/murfey/client/contexts/tomo.py

View check run for this annotation

Codecov / codecov/patch

src/murfey/client/contexts/tomo.py#L168 

Added line #L168 was not covered by tests
else:
logger.info(
Copy link
Contributor

@tieneupin tieneupin Mar 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this message need to pop up on the TUI/server log (levels info and above will appear, based on Murfey's current settings)? It won't appear if we switch this to logger.debug()

Copy link
Contributor Author

@stephen-riggs stephen-riggs Mar 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's probably worth recording this to the TUI, as it shows that processing hasn't started yet

"Cannot register data collection yet as no values from mdoc"
)
else:
logger.info(
"Cannot register data collection yet as no values from mdoc"
)

except Exception as e:
logger.error(f"ERROR {e}, {environment.data_collection_parameters}")
Expand Down