Skip to content

Commit

Permalink
Updating the extensions documentation.
Browse files Browse the repository at this point in the history
Discussion of an insidious file corruption situation (coarse images at
the apex of the pyramid are valid, finer images, close to the pyramid
base are corrupt). Happened when converting from original microscopy
format to imaris using network or external drives.
  • Loading branch information
zivy committed Jul 10, 2023
1 parent bc63d8b commit fa6ca98
Show file tree
Hide file tree
Showing 4 changed files with 39 additions and 3 deletions.
12 changes: 10 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,17 @@ Z. Yaniv, B. Lowekamp, "SimpleITK Imaris Extensions", doi: [10.5281/zenodo.78540
## Extensions Listing
> :warning: **Corrupt files** will cause the extensions to fail with an error message "*...OSError: Can't read data (inflate() failed)*". In some cases imaris is able to read such files while the extensions fail to do so. A solution, that often works, is to read the file into imaris and then "Save as" to a new file which can then be read by the extensions.
---
> :warning: **Out of memory errors** will cause the extensions to fail with an error message along the lines of "*...Failed to allocate memory for image.*". The minimal RAM size required to run an extension depends on the image sizes and the specific extension in use. For common image sizes, **16GB of RAM** is often sufficient, **64GB or more** is desirable. If memory size is not sufficient, consider increasing the size of the machine's [virtual memory](https://en.wikipedia.org/wiki/Virtual_memory). Experience has shown us that some extensions, e.g. `XTChannelArithmetic`, do work on systems with only **8GB of RAM** when configured appropriately (using a memory efficient slice-by-slice processing at the cost of longer runtimes).
**WARNINGS**
Avoid converting files into imaris format using a network or external drive, this has the potential to produce corrupt files that are hard to identify as such (unless you enjoy hours of debugging). This issue is not specific to the work found here. For more details see the [XTRegisterSameChannel documentation](http://niaid.github.io/imaris_extensions/XTRegisterSameChannel.html).
**Corrupt files** will cause the extensions to fail with an error message "*...OSError: Can't read data (inflate() failed)*". In some cases imaris is able to read such files while the extensions fail to do so. A solution, that often works, is to read the file into imaris and then "Save as" to a new file which can then be read by the extensions.
**Out of memory errors** will cause the extensions to fail with an error message along the lines of "*...Failed to allocate memory for image.*". The minimal RAM size required to run an extension depends on the image sizes and the specific extension in use. For common image sizes, **16GB of RAM** is often sufficient, **64GB or more** is desirable. If memory size is not sufficient, consider increasing the size of the machine's [virtual memory](https://en.wikipedia.org/wiki/Virtual_memory). Experience has shown us that some extensions, e.g. `XTChannelArithmetic`, do work on systems with only **8GB of RAM** when configured appropriately (using a memory efficient slice-by-slice processing at the cost of longer runtimes).
---
### Algorithms
1. [XTRegisterSameChannel](http://niaid.github.io/imaris_extensions/XTRegisterSameChannel.html) - Registration of 2D or 3D images that share a common channel (correlation based affine alignment). Sample datasets are freely available on zenodo [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.4632320.svg)](https://doi.org/10.5281/zenodo.4632320). A video illustrating the usage of the extension is available on [YouTube](https://www.youtube.com/watch?v=rrCajI8jroE).
Expand Down
26 changes: 26 additions & 0 deletions XTRegisterSameChannel.py
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,32 @@ class RegisterSameChannelDialog(ieb.ImarisExtensionBase):
is projected along the z-direction using a mean projection, affine registration is performed in
2D and then the z-stacks are aligned in the z-direction so that their centers are aligned.
Results with artifacts
++++++++++++++++++++++
In some cases we have observed that registration appears to work, but the final result contains
strange artifacts even though the input images **appear to be visually valid**. An example result
with artifacts is shown below (image courtesy Dr. Andrea Radtke):
.. figure:: docs/images/registration_result_from_courrpt_input.jpeg
:alt: registration result from corrupt input
This is not a problem with the registration or resampling, it is a problem with the input. The original
image is already corrupt. We have identified that this happens when the image is imported into imaris
from its original file format and the resulting imaris file is saved to a network or external drive.
**Copy the original image to the local drive, import into imaris and save to local drive, problem solved.**
| I want to know more:
| **Q**: Why does my original image appear to be visually valid and yet you claim it is corrupt?
| **A**: The imaris file format is hierarchical, an image pyramid. When we open an image in imaris we are looking
at a low resolution version of the image, close to the apex of the pyramid. These images are usually
very small and are saved correctly over the network. When we zoom into the image, the imaris viewer will
switch to a higher resolution version of the image from the pyramid. The closer we get to the base the
larger the image and the higher the chance that it be corrupted when saving over the network. The registration
uses the base image (highest resolution) and resamples it, and thus exposes that the image is
corrupt.
| Don't trust us? If your registration result exhibits artifacts, open your original image in imaris and
zoom in, the artifacts will "magically" appear.
""" # noqa

def __init__(self):
Expand Down
4 changes: 3 additions & 1 deletion docs/generate_documentation.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
with open(os.path.join(tmpdirname, f_name[:-3] + ".rst"), "w") as fp:
fp.write(inspect.getdoc(getattr(module, attribute_name)))
rst_content = inspect.getdoc(getattr(module, attribute_name))
# update all paths to images and write to file
fp.write(rst_content.replace("docs/images", "../docs/images"))
fp.flush()
os.system(f"pandoc -s {fp.name} -o {f_name[:-3]}.html -c {css_file_name}")
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit fa6ca98

Please sign in to comment.