Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update for latest version BigStitcher, add fusion (BDV-playground) #39

Open
wants to merge 32 commits into
base: devel
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
e870bea
Add method for dataset definition for angles and illuminations
lguerard Feb 11, 2025
77138ff
Start WIP for refactor using latest BigStitcher
lguerard Feb 11, 2025
c5528dd
Add method for fusion using BDV Playground
lguerard Feb 11, 2025
e1845e3
Refactor to work with latest BigStitcher
lguerard Mar 6, 2025
9aa57db
Add method for fusion using BIOP Kheops to OME-TIFF
lguerard Mar 6, 2025
b47ab1b
Use the multi view dataset definition command instead
lguerard Mar 11, 2025
4233176
Update tests to latest modifications
lguerard Mar 11, 2025
cdbcaf4
Update method for fusion using BDV playground
lguerard Mar 11, 2025
28b319e
Add method for associating label images using 3DImageJSuite
lguerard Mar 11, 2025
aba06fa
Fix the filtering of objects to use the calibrated method
lguerard Mar 11, 2025
2c58ea9
Use the morpholibj package to 2D dilate labels
lguerard Mar 11, 2025
2ab85e9
Format imports
lguerard Mar 11, 2025
07fa81b
Add method to write results to CSV
lguerard Mar 11, 2025
1ec6f37
Add method to return the median value of a list
lguerard Mar 11, 2025
7fffba7
Add methods for 3D Maxima Finder and 3D Watershed
lguerard Mar 11, 2025
d2bfd09
Update tests to work with Multi-View dataset definition
lguerard Mar 11, 2025
44a77fe
Add processing library to do basic methods
lguerard Mar 11, 2025
af09a4b
Revert "Add processing library to do basic methods"
lguerard Mar 12, 2025
1294ede
Revert "Add method to return the median value of a list"
lguerard Mar 12, 2025
f160c59
Revert "Format imports"
lguerard Mar 12, 2025
dc87aad
Revert "Use the morpholibj package to 2D dilate labels"
lguerard Mar 12, 2025
dc3382e
Revert "Fix the filtering of objects to use the calibrated method"
lguerard Mar 12, 2025
93dadc9
Revert "Add method for associating label images using 3DImageJSuite"
lguerard Mar 12, 2025
201a6c7
Add missing check function
lguerard Mar 12, 2025
af3616a
Fix docstring to reflect real string
lguerard Mar 12, 2025
ab8e5aa
Formatting
lguerard Mar 12, 2025
5cc36d3
Fix string
lguerard Mar 12, 2025
678596e
Fix string
lguerard Mar 12, 2025
09c790c
Use same input for test to work
lguerard Mar 12, 2025
c03f792
Formatting
lguerard Mar 12, 2025
8f67cb0
Fix input for angle to only support input options
lguerard Mar 12, 2025
7c63de8
Merge branch 'devel' into laurent/devel
ehrenfeu Mar 20, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
155 changes: 125 additions & 30 deletions src/imcflibs/imagej/bdv.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@
import shutil
import sys

from ch.epfl.biop.scijava.command.spimdata import (
FuseBigStitcherDatasetIntoOMETiffCommand,
)
from ij import IJ

from .. import pathtools
Expand Down Expand Up @@ -198,8 +201,8 @@
range_end : int, optional
Contains the end of the range, by default None.

Notes:

Check failure on line 204 in src/imcflibs/imagej/bdv.py

View workflow job for this annotation

GitHub Actions / Ruff ⚡🕵

src/imcflibs/imagej/bdv.py:204:9: D406 Section name should end with a newline ("Notes")
------

Check failure on line 205 in src/imcflibs/imagej/bdv.py

View workflow job for this annotation

GitHub Actions / Ruff ⚡🕵

src/imcflibs/imagej/bdv.py:205:9: D409 Section underline should match the length of its name ("Notes")
Previous function name : angle_select().
"""

Expand All @@ -224,8 +227,8 @@
range_end : int, optional
Contains the end of the range, by default None.

Notes:

Check failure on line 230 in src/imcflibs/imagej/bdv.py

View workflow job for this annotation

GitHub Actions / Ruff ⚡🕵

src/imcflibs/imagej/bdv.py:230:9: D406 Section name should end with a newline ("Notes")
------

Check failure on line 231 in src/imcflibs/imagej/bdv.py

View workflow job for this annotation

GitHub Actions / Ruff ⚡🕵

src/imcflibs/imagej/bdv.py:231:9: D409 Section underline should match the length of its name ("Notes")
Previous function name : channel_select().
"""

Expand All @@ -250,8 +253,8 @@
range_end : int, optional
Contains the end of the range, by default None.

Notes:

Check failure on line 256 in src/imcflibs/imagej/bdv.py

View workflow job for this annotation

GitHub Actions / Ruff ⚡🕵

src/imcflibs/imagej/bdv.py:256:9: D406 Section name should end with a newline ("Notes")
------

Check failure on line 257 in src/imcflibs/imagej/bdv.py

View workflow job for this annotation

GitHub Actions / Ruff ⚡🕵

src/imcflibs/imagej/bdv.py:257:9: D409 Section underline should match the length of its name ("Notes")
Previous function name : illumination_select().
"""

Expand All @@ -276,8 +279,8 @@
range_end : int, optional
Contains the end of the range, by default None.

Notes:

Check failure on line 282 in src/imcflibs/imagej/bdv.py

View workflow job for this annotation

GitHub Actions / Ruff ⚡🕵

src/imcflibs/imagej/bdv.py:282:9: D406 Section name should end with a newline ("Notes")
------

Check failure on line 283 in src/imcflibs/imagej/bdv.py

View workflow job for this annotation

GitHub Actions / Ruff ⚡🕵

src/imcflibs/imagej/bdv.py:283:9: D409 Section underline should match the length of its name ("Notes")
Previous function name : tile_select().
"""

Expand All @@ -302,8 +305,8 @@
range_end : int, optional
Contains the end of the range, by default None.

Notes:

Check failure on line 308 in src/imcflibs/imagej/bdv.py

View workflow job for this annotation

GitHub Actions / Ruff ⚡🕵

src/imcflibs/imagej/bdv.py:308:9: D406 Section name should end with a newline ("Notes")
------

Check failure on line 309 in src/imcflibs/imagej/bdv.py

View workflow job for this annotation

GitHub Actions / Ruff ⚡🕵

src/imcflibs/imagej/bdv.py:309:9: D409 Section underline should match the length of its name ("Notes")
Previous function name : timepoint_select().
"""

Expand Down Expand Up @@ -506,7 +509,7 @@
>>> opts.fmt_acitt_options()
... multiple_angles=[NO (one angle)]
... multiple_channels=[YES (all channels in one file)]
... multiple_illuminations=[NO (one illumination direction)]
... multiple_illuminations_directions=[NO (one illumination direction)]
... multiple_tiles=[YES (all tiles in one file)]
... multiple_timepoints=[NO (one time-point)]
"""
Expand Down Expand Up @@ -543,15 +546,43 @@
"multi_multi": MULTI_MULTI_FILE,
}

def check_definition_option_ang_ill(self, value):
"""Check if the value is a valid definition option.

This is needed for angles and illuminations because support is not
available for multiple angles and illuminations in a single file.

Parameters
----------
value : str
Entered value by the user.

Returns
-------
dict(str, str): dictionary containing the correct string definition.
"""
if value not in [
"single",
"multi_multi",
]:
raise ValueError(
"Value must be one of single, multi_multi. Support for multi_single is not available for angles and illuminations"
)

return {
"single": SINGLE_FILE,
"multi_multi": MULTI_MULTI_FILE,
}

def set_angle_definition(self, value):
"""Set the value for the angle definition

Parameters
----------
value : str
One of `single`, `multi_single` or `multi_multi`.
One of `single` or `multi_multi`.
"""
choices = self.check_definition_option(value)
choices = self.check_definition_option_ang_ill(value)
self._angle_definition = choices[value] % "angle"
log.debug("New 'angle_definition' setting: %s", self._angle_definition)

Expand All @@ -575,7 +606,7 @@
value : str
One of `single`, `multi_single` or `multi_multi`.
"""
choices = self.check_definition_option(value)
choices = self.check_definition_option_ang_ill(value)

Check warning on line 609 in src/imcflibs/imagej/bdv.py

View check run for this annotation

Codecov / codecov/patch

src/imcflibs/imagej/bdv.py#L609

Added line #L609 was not covered by tests
self._illumination_definition = choices[value] % "illumination direction"
log.debug(
"New 'illumination_definition' setting: %s", self._illumination_definition
Expand Down Expand Up @@ -609,7 +640,7 @@
"""Format Angle / Channel / Illumination / Tile / Timepoint options.

Build a string providing the `multiple_angles`, `multiple_channels`,
`multiple_illuminations`, `multiple_tiles` and `multiple_timepoints` options
`multiple_illuminations_directions`, `multiple_tiles` and `multiple_timepoints` options
that can be used in a BDV-related `IJ.run` call.

Returns
Expand All @@ -619,7 +650,7 @@
parameters = [
"multiple_angles=" + self._angle_definition,
"multiple_channels=" + self._channel_definition,
"multiple_illuminations=" + self._illumination_definition,
"multiple_illuminations_directions=" + self._illumination_definition,
"multiple_tiles=" + self._tile_definition,
"multiple_timepoints=" + self._timepoint_definition,
]
Expand Down Expand Up @@ -807,23 +838,29 @@
angle_rotation = ""

options = (
"define_dataset=[Automatic Loader (Bioformats based)] "
"define_dataset=[Automatic Loader (Bioformats based)]"
+ " "
+ "project_filename=["
+ project_filename
+ ".xml"
+ "] "
+ "path=["
+ file_info["path"]
+ file_info["full"]
+ "] "
+ "exclude=10 "
# + "bioformats_series_are?="
# + bf_series_type
# + " "
+ "bioformats_series_are?="
+ bf_series_type
+ " "
+ "move_tiles_to_grid_(per_angle)?=[Do not move Tiles to Grid (use Metadata if available)] "
+ "how_to_load_images=["
+ "how_to_store_input_images=["
+ resave
+ "] "
+ "dataset_save_path=["
+ "load_raw_data_virtually"
+ " "
+ "metadata_save_path=["
+ dataset_save_path
+ "] "
+ "image_data_save_path=["
+ dataset_save_path
+ "] "
+ "check_stack_sizes "
Expand All @@ -836,21 +873,11 @@
+ " "
+ "setups_per_partition=0 "
+ "use_deflate_compression "
# + "export_path=["
# + dataset_save_path
# + "]",
)

log.debug(options)

if bf_series_type == "Tiles":
log.debug("Doing tiled dataset definition")
IJ.run("Define dataset ...", str(options))
elif bf_series_type == "Angles":
log.debug("Doing multi-view dataset definition")
IJ.run("Define Multi-View Dataset", str(options))
else:
raise ValueError("Wrong answer for series type")
IJ.run("Define Multi-View Dataset", str(options))


def define_dataset_manual(
Expand Down Expand Up @@ -892,6 +919,7 @@
+ "project_filename=["
+ xml_filename
+ "] "
+ "_____"
+ definition_opts.fmt_acitt_options()
+ " "
+ "image_file_directory="
Expand All @@ -903,11 +931,11 @@
+ " "
+ "calibration_type=[Same voxel-size for all views] "
+ "calibration_definition=[Load voxel-size(s) from file(s)] "
+ "imglib2_data_container=[ArrayImg (faster)]"
# + "imglib2_data_container=[ArrayImg (faster)]"
)

log.debug("Manual dataset defintion options: <%s>", options)
IJ.run("Define dataset ...", str(options))
log.debug("Manual dataset definition options: <%s>", options)
IJ.run("Define Multi-View Dataset", str(options))

Check warning on line 938 in src/imcflibs/imagej/bdv.py

View check run for this annotation

Codecov / codecov/patch

src/imcflibs/imagej/bdv.py#L937-L938

Added lines #L937 - L938 were not covered by tests


def resave_as_h5(
Expand Down Expand Up @@ -1424,7 +1452,9 @@
downsampling=1,
interpolation="[Linear Interpolation]",
pixel_type="[16-bit unsigned integer]",
fusion_type="Avg, Blending",
export="HDF5",
compression="Zstandard",
):
"""Call BigStitcher's "Fuse Dataset" command.

Expand Down Expand Up @@ -1475,11 +1505,13 @@
+ "interpolation="
+ interpolation
+ " "
+ "fusion_type=["
+ fusion_type
+ "] "
+ "pixel_type="
+ pixel_type
+ " "
+ "interest_points_for_non_rigid=[-= Disable Non-Rigid =-] "
+ "blend "
+ "preserve_original "
+ "produce=[Each timepoint & channel] "
)
Expand All @@ -1506,9 +1538,12 @@

options = (
options
+ "fused_image=[ZARR/N5/HDF5 export using N5-API] "
+ "fused_image=[OME-ZARR/N5/HDF5 export using N5-API] "
+ "define_input=[Auto-load from input data (values shown below)] "
+ "export=HDF5 "
+ "compression="
+ compression
+ " "
+ "create "
+ "create_0 "
+ "hdf5_file=["
Expand All @@ -1527,4 +1562,64 @@
)

log.debug("Dataset fusion options: <%s>", options)
IJ.run("Fuse dataset ...", str(options))
IJ.run("Image Fusion", str(options))

Check warning on line 1565 in src/imcflibs/imagej/bdv.py

View check run for this annotation

Codecov / codecov/patch

src/imcflibs/imagej/bdv.py#L1565

Added line #L1565 was not covered by tests


def fuse_dataset_bdvp(
project_path,
command,
processing_opts=None,
result_path=None,
compression="LZW",
):
"""Export a BigDataViewer project using the BIOP Kheops exporter.

This function uses the BIOP Kheops exporter to convert a BigDataViewer project into a
OME-TIFF files, with optional compression.

Parameters
----------
project_path : str
Full path to the BigDataViewer XML project file.
command : CommandService
The Scijava CommandService instance to execute the export command.
processing_opts : ProcessingOptions, optional
Options defining which parts of the dataset to process. If None, default processing
options will be used (process all angles, channels, etc.).
result_path : str, optional
Path where to store the exported files. If None, files will be saved in the same
directory as the input project.
compression : str, optional
Compression method to use for the TIFF files. Default is "LZW".

Notes
-----
This function requires the PTBIOP update site to be enabled in Fiji/ImageJ.
"""
if processing_opts is None:
processing_opts = ProcessingOptions()

Check warning on line 1600 in src/imcflibs/imagej/bdv.py

View check run for this annotation

Codecov / codecov/patch

src/imcflibs/imagej/bdv.py#L1599-L1600

Added lines #L1599 - L1600 were not covered by tests

file_info = pathtools.parse_path(project_path)
if not result_path:
result_path = file_info["path"]

Check warning on line 1604 in src/imcflibs/imagej/bdv.py

View check run for this annotation

Codecov / codecov/patch

src/imcflibs/imagej/bdv.py#L1602-L1604

Added lines #L1602 - L1604 were not covered by tests
# if not os.path.exists(result_path):
# os.makedirs(result_path)

command.run(

Check warning on line 1608 in src/imcflibs/imagej/bdv.py

View check run for this annotation

Codecov / codecov/patch

src/imcflibs/imagej/bdv.py#L1608

Added line #L1608 was not covered by tests
FuseBigStitcherDatasetIntoOMETiffCommand,
True,
"image",
project_path,
"output_dir",
result_path,
"compression",
compression,
"subset_channels",
"",
"subset_slices",
"",
"subset_frames",
"",
"compress_temp_files",
False,
)
31 changes: 31 additions & 0 deletions src/imcflibs/imagej/misc.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
"""Miscellaneous ImageJ related functions, mostly convenience wrappers."""

import csv
import sys
import time
import smtplib
Expand Down Expand Up @@ -417,3 +418,33 @@
threshold_value = int(round(threshold_value.get()))

return threshold_value


def write_results(out_file, content):
"""Write the results to a csv file.

Parameters
----------
out_file : str
Path to the output file.
content : list of OrderedDict
List of dictionaries representing the results.

"""

# Check if the output file exists
if not os.path.exists(out_file):

Check warning on line 436 in src/imcflibs/imagej/misc.py

View check run for this annotation

Codecov / codecov/patch

src/imcflibs/imagej/misc.py#L436

Added line #L436 was not covered by tests
# If the file does not exist, create it and write the header
with open(out_file, "wb") as f:
dict_writer = csv.DictWriter(

Check warning on line 439 in src/imcflibs/imagej/misc.py

View check run for this annotation

Codecov / codecov/patch

src/imcflibs/imagej/misc.py#L438-L439

Added lines #L438 - L439 were not covered by tests
f, content[0].keys(), delimiter=";"
)
dict_writer.writeheader()
dict_writer.writerows(content)

Check warning on line 443 in src/imcflibs/imagej/misc.py

View check run for this annotation

Codecov / codecov/patch

src/imcflibs/imagej/misc.py#L442-L443

Added lines #L442 - L443 were not covered by tests
else:
# If the file exists, append the results
with open(out_file, "ab") as f:
dict_writer = csv.DictWriter(

Check warning on line 447 in src/imcflibs/imagej/misc.py

View check run for this annotation

Codecov / codecov/patch

src/imcflibs/imagej/misc.py#L446-L447

Added lines #L446 - L447 were not covered by tests
f, content[0].keys(), delimiter=";"
)
dict_writer.writerows(content)

Check warning on line 450 in src/imcflibs/imagej/misc.py

View check run for this annotation

Codecov / codecov/patch

src/imcflibs/imagej/misc.py#L450

Added line #L450 was not covered by tests
Loading
Loading